NeuroGeMS: Multimodal Data Science for AI

KEDRI at AUT has developed the world first development system for the analysis and predictive modelling of multimodal data, including multiple and heterogeneous static, vector based data and spatio-temporal data, named NeuroGeMS. KEDRI has secured more than $2.1m to develop new data science technology for mental health diagnosis via the Catalyst: Strategic – New Zealand-Singapore Data Science Research Programme, supported by the New Zealand Ministry of Business Innovation and Employment and Singapore Data Science Consortium.

Various structured and unstructured data can be integrated in one model, to better understand and predict outputs.  Different strategies can be utilised, such as: early integration; late integration; intermediate and combined integration. AI applications include: understanding and predicting personalised medical events (NZ, Singapore); environmental predictive modelling (NZ).

Aotearoa New Zealand has one of the highest prevalence rates of depression worldwide. It accounts for half of the annual suicides and attempted suicides, particularly among 13-25 years old. There is a need to develop new data science methods for accurate diagnosis/prognosis of mental illness and suggest optimal interventions.

The research focuses on developing a new computational neuro-genetic modelling based on machine-learning/AI methods for diagnosis of mental health issues. This study boasts an unprecedented level of data varieties, including cutting edge genomics, proteomics and metabonomic technologies.

This project is jointly coordinated via AUT's KEDRI and our Singapore research partner from the School of Biological Sciences, Nanyang Technological University .

Main outcomes

  • Development of new machine-learning/AI methods for multimodal data modelling.
  • Better clinical intervention via early and accurate diagnosis of mental health issues in at-risk youth.
  • Developing personalized modelling for a better understanding of individual factors that trigger mental illnesses.

Key features

  1. Multimodal Fusion: Effortlessly integrate data from diverse sources and modalities. NeuroGeMs  empowers you to harness the full potential of your data and extract valuable insights through advanced fusion techniques.
  2. Interpretability: Gain deeper understanding and build trust in your models with NeuroGeMs 's comprehensive interpretability module. Easily analyse and explain model decisions, making your AI applications more transparent and ethically sound.
  3. Efficient Dataset Management: Organise and manage your datasets like a with NeuroGeMs' user-friendly tab for handling data. Seamlessly import, clean, and preprocess data to ensure the highest data quality for your experiments.
  4. Intuitive Model Creation: NeuroGeMs provides a smooth workflow to create, customise and fine-tune machine learning models, accommodating both beginners and seasoned data scientists.
  5. Streamlined Experimentation: Run, monitor, and save experiments with ease using NeuroGeMs 's smart experiment management feature. Spend less time on logistics and more time on innovation.
  6. Data Visualisation at Your Fingertips: Visualize your data and model performance with NeuroGeMs 's interactive data visualisation tools. Gain valuable insights and communicate your findings more effectively.

Image of a laptop

Budhraja, S., Singh, B., Tan, S., Doborjeh, M., Doborjeh, Z., Lai, E., ... & Kasabov, N. (2024, December). NeuroGeMS: An Open-Source GUI Software for Multimodal Modelling in Biomedical Research and Applications. In International Conference on Neural Information Processing (pp. 218-232). Singapore: Springer Nature Singapore.

Project members

  • Nikola Kasabov, AUT
  • Sugam Budhraja, AUT
  • Balkaran Singh, AUT
  • Edmund Lai, AUT
  • Zohreh Doborjeh, AUT
  • Margaret Hinepo Williams, AUT
  • Alex Sumich, Nottingham Trent University
  • Wilson Wen Bin Goh, Nanyang Technological University
  • Jimmy Lee,  Institute of Mental Health (IMH) Singapore
  • Limsoon Wong, National University of Singapore