AI-CENTIVE is all about using AI to understand and predict local mobility, with the use of incentives to encourage more use of sustainable mobility options. One of the cases where it is beneficial to leave the private car at home and use either shared transportation (e.g. ridesharing, public transport) or a more flexible form of mobility (e.g. bicycle, scooter) is when a large and important event is taking place near to a citizen’s usual route which would affect traffic fluidity. This can be seen as an opportunity to promote these alternative forms of mobility as long as relevant events can be identified in advance and the locations affected by them. 

The Web dashboard of AI-CENTIVE (from project partner webLyzard) is a tool to identify and track the effects of large events on mobility within Austria. MODUL Technology performs the data collection from the Web to feed the dashboard’s analytics and visualization components. We demonstrate it with respect to two concerts which are attracting large numbers of visitors to Vienna. Firstly, there were the planned Taylor Swift concerts which were cancelled but led alternatively to large groups meeting in various locations in the city. Then there is the series of concerts by Coldplay planned for 21-25 August in the Ernst Happel Stadium. Through both, we can reflect on how the dashboard – which provides “Web intelligence”, meaning an analysis of the content of public Web pages such as news articles – reveals details of mobility patterns for both past and future events. 

The Taylor Swift concerts were planned for 8-10 August. Considering Web content over that period as many Swift fans still came to Vienna (7-11 August, searching for “Swift” as well as “Swifties”), the top story extracted from the online news reports refers to the Corneliusgasse together with the English “Cornelia Street” which is the title of a Taylor Swift song. Because of this song, many Swifties gathered in the Viennese equivalent in the sixth district. Meanwhile, the geo visualisation (related to locations extracted from the text) in the dashboard reflects a concentration of Swift concert related news mentioning Stephansplatz, where thousands of fans gathered in the evenings to sing Swift songs. 

The top story shown in the dashboard for news over 7-11 August searching for “Swift” or “Swifties” references Corneliusgasse.

Vienna geo-visualisation with a concentration of news articles mentioning Stephansplatz

This analysis over past news shows how key locations for events that have occurred can be identified. Instead of the Ernst Happel Stadium (as would have been expected if we had been looking into the future), we see that it was Stephansplatz and (perhaps the least expected) a short street in the sixth district called Corneliusgasse which are mentioned the most. Indeed, we can validate this data from the dashboard as correct, knowing that indeed these were the two most popular locations in Vienna for Swifties to gather over the weekend of the (cancelled) concerts. 

Having shown how the data analysis and visualisations by the dashboard can correctly highlight locations that are affected by mobility-relevant events, we can make use of the predictive mode to identify and make similar mobility-related predictions for future events. MODUL Technology developed text-based predictive analytics originally for the ReTV project (to identify future topics of interest to TV audiences). Here, online content is selected based on reference to future dates and its analysis is presented in the same way (with visualisations) in the dashboard. We choose 20-26 August for our future date range (this experiment was conducted on 19 August) and analyse all public news articles in Austria which mention any of those dates. The dashboard’s “storygraph” visualisation provides a chronological overview of detected events. Interestingly we are alerted to the “Oper im Steinbruch St Margarethen” which is also mentioned during this period. By drilling down to only the Coldplay documents, we can see in the same storygraph visualisation that they are associated to the period 21-25 August.

(above) Predictions for 24-25 Aug in the storygraph visualisation with Steinbruch opera and the Coldplay concerts. (below) Coldplay concerts cover the period 21-25 Aug (note that 20 and 26 Aug are empty). 

Looking again at locations associated with the event in the online documents, (Ernst) Happel Stadium stands out as being mentioned in almost every matching document. Therefore we could automatically extract this data via the dashboard that this is the primary location for the Coldplay concerts. From the geo visualisation we find also a reference to Ludwig Koessler Platz, and it turns out a bus which runs from there will be diverted for the concerts so we have found another mobility-relevant association. 

(above) “Happel Stadion” as the primary location mentioned together with the Coldplay concerts. (below) Ludwig Koessler Platz is also mentioned (and mobility there is affected by the concerts). 

The case of Taylor Swift is perhaps a rare one, where concerts are cancelled at short notice, but serve as a reminder that predictions are always subject to other circumstances we can not anticipate. However, as AI-CENTIVE seeks to anticipate future states where private mobility is affected in order to incentivize alternatives which are more sustainable, the Web based dashboard can be a tool for stakeholders to identify those events that will potentially affect local mobility. The main locations that will be affected and the time period in which they are affected can be predicted too, enabling incentives to be made to local citizens in a timely manner to choose a better form of mobility which contributes to saving CO2. 

AI-CENTIVE has already helped save more than 100,000 kg of CO2 and with further innovations such as this event-based prediction, we plan to contribute to even more savings in the future. MODUL Technology provides AI-based predictions of future mobility to guide incentive schemes and promote more sustainable mobility choice.

For more information about AI-CENTIVE, see the project website.

The FFG funded GENTIO project has ended with MODUL Technology contributing advances in AI-based NLP and NER/NEL approaches, development of its Semantic Knowledge Base, and new classification and summarization algorithms. State of the art AI has improved our Natural Language Processing, Named Entity Recognition and Named Entity Linking components. Good NER combined with relation extraction can feed our Semantic Knowledge Base with new entities and relations automatically identified in news articles. To explore new possibilities to tackle this, we started looking at fine tuning Large Language Models (LLMs) with promising early results, the subject of a conference presentation. Transformers models were compared for the task of text summarization.
A significant impact was made in the task of news article classification, following the IPTC News categories. Here, we decided on fine tuning on top of BERTopic, a neural model for topic classification. The initial topic classification is fed through a Transformer model to align topics to the IPTC News categories. We could show higher accuracy scores on annotated news corpora than just using a language model like RoBERTa to classify articles directly, also published this year at a conference. GENTIO has enabled MODUL Technology to advance in its deployment of state of the art AI (e.g. Transformers models, language models) for core text understanding tasks than underlie ‘Web intelligence’, i.e. extracting trends and insights from Web data.

The FFG funded project EPOCH, coordinated by MODUL Technology, demonstrated the groundbreaking use of machine learning/AI approaches to time series forecasting combined with Web intelligence – the analysis of topics and trends in online news and social media over time. Supported by webLyzard technology’s Web intelligence platform and wood price data from KPMG, MODUL Technology’s AI experts combined statistical forecasting models with features extracted from the online news and social media based on mentions of terms relevant to the wood market and combined sentiment of documents mentioning the terms (e.g. ‘Borkenkäfer’, the bark beetle which damages trees).  Four categories of terms were created, and were tested for their predictive power (the extent to which they improve the accuracy of price prediction compared to pure statistical forecasting). We compared a number of models (Random Forest, XGBoost, LSTM, GRU) with four different datasets (hardwood, softwood, nadelholz (coniferous), rohpapier (raw paper)). The rolling mean sentiment feature was found to be the most predictive for future wood prices, especially for the terms related to weather and pests. We provide a slideset explaining the wood price prediction experiments and a Web interface which allows you to visualise the accuracy and feature importance in prediction using different models and datasets. The experiment code is also available to test the models with your own price data. 

• Slides
• Web interface
• Experiment code

This groundbreaking work was extended to look at predictions of prices in other markets and make use of recent Transformers models such as FinBERT to generate the sentiment features from a news corpus. The developed models outperformed classical models, showcasing the fact that combining basic economic logic (e.g., supply and demand) with affective and background knowledge can lead to significant improvements. MODUL Technology will continue to bring this advance in price prediction to other domains through future projects. 

Journal Articles

• Kaplan, H., Weichselbraun, A., Braşoveanu, A.M.P. (2023) Integrating Economic Theory, Domain Knowledge and Social Knowledge into Hybrid Sentiment Models for Predicting the Crude Oil Markets. Cognitive Computation. Springer Nature. DOI: 10.1007/s12559-023-10129-4.
• Weichselbraun, A., Steixner, J., Brasoveanu, A.M.P., Scharl, A., Gobel, M & Nixon, L.B. (2022). Automatic Expansion of Domain-Specific Affective Models for Web Intelligence Applications. Cognitive Computation 14(1), pp. 228-245. DOI: 10.1007/s12559-021-09839-4.

Book Chapters

• Braşoveanu, A M.P., & Andonie, R. (2022). Visualizing and Explaining Language Models. In Integrating Artificial Intelligence and Visualization for Visual Knowledge Discovery, pp. 213-237. Cham: Springer International Publishing. DOI: 10.1007/978-3-030-93119-3_8.

Modul Technology is excited to start working on the new project AI-CENTIVE, funded by the FFG via the ICT of the Future program. The goal of the 3 year project is to build and manage a complex mobility data ecosystem which will support changing citizens’ mobility behaviour towards choosing more sustainable options. It is an interdisciplinary project that combines data science, deep learning, weather prediction, information visualisation and tourism management. The project is implemented in collaboration with webLyzard technology, Zentralanstalt für Meteorologie und Geodynamik (ZAMG), ummadum, Data Intelligence Initiative (DIO) and BOKU. 

Nowadays the data on mobility choices and the context in which they are taken is split. The aim of the project is to turn it into actionable information thanks to efficient knowledge extraction. A common mobility data ecosystem will merge this data, allowing new AI models that learn to anticipate mobility choices based on varying contexts such as the availability of more environmentally friendly options. Customised incentives leveraging the AI predictions will motivate citizens to make more sustainable mobility choices. As such, AI-CENTIVE will contribute to Austria’s achievement of the UN 2030 Agenda for Sustainable Development. 

Stay tuned … we will very soon go online with the project’s website!

MODUL Technology is excited to announce that its participation in the TRANSMIXR project has begun,  backed by EU Horizon Europe funding of 9 million euros.

With the maturity of eXtended Reality (XR) and Artificial Intelligence (AI) technologies, a unique window of opportunity for the European Creative and Cultural Sector (CCS) exists to reimagine digital co-creation, interaction and engagement possibilities. The TRANSMIXR platform will provide (i) a distributed XR Creation Environment that supports remote collaboration practices, and (ii) an XR Media Experience Environment for the delivery and consumption of evocative and immersive media experiences. Ground-breaking AI techniques for the understanding and processing of complex media content will enable the reuse of heterogeneous assets across immersive content delivery platforms. Using the Living Labs methodology, TRANSMIXR will develop and evaluate four pilots that bring the vision of future media experiences to life in four important CCS domains: news media, broadcasting, performing arts, and cultural heritage.

TRANSMIXR Project Coordinator, Dr. Niall Murray of TUS and investigator in the SFI Adapt Centre said “TRANSMIXR is a very exciting project that will create a suite of user centric technologies to support the creation, consumption and understanding of new media experiences in distributed, collaborative and immersive ways. Underpinned by the convergence of AI and XR, the design of these new systems will be informed by and be evaluated with real end users. A key strength of the TRANSMIXR consortium is its interdisciplinary nature, bringing complimentary technical, methodological and domain expertise together to create impactful solutions for the creative and culture sectors.”

MODUL Technology CTO and project lead, Dr. Lyndon Nixon, added “TRANSMIXR will benefit immediately from the multilingual Web data collection and annotation pipeline of MODUL Technology, while we look forward to the opportunity to further innovate through AI in terms of content understanding and semantic search across heterogeneous, multimodal data collections.”

Website: https://www.transmixr.eu/

In AI4Green, the latest program for research funding in Austria managed by the FFG, MODUL Technology is excited to begin a new funded project called SDG-HUB in which it will build a Knowledge Graph for the Sustainable Development Goals (SDGs) in Austria.

Having started in May 2022, the three-year project’s goal is to support Austria’s achievement of the SDGs of the Agenda 2030 through building a knowledge repository with AI-based semantic search and visualization capabilities. The repository will offer concrete insights into citizen and stakeholder perceptions via monitoring of the online discourse as well as measure progress towards achieving the Agenda 2030 and Paris Agreement on climate change mitigation via online data ingestion and analysis. Through a better understanding of the social mood and environmental indicators, national research and scientific communities will be enabled to communicate more effectively to citizens about both proper and improper behavior and contribute to measurable improvements in achieving climate and sustainability goals in Austria.

Together with the Zentralanstalt für Meteorologie und Geodynamik (ZAMG), the Climate Change Centre Austria (CCCA), the University of Innsbruck as well as project coordinator and long-time collaborator webLyzard technology, MODUL Technology provides competencies in Automatic Knowledge Graph Construction (AKGC), Word Sense Disambiguation (WSD) and relation extraction (Slot Filling). These will be used to build an SDG-HUB Knowledge Graph that provides disambiguated identities for all concepts that form part of the SDG discussion (particularly national and regional concepts that would be missing from global graphs such as DBPedia or Wikidata). Complemented by facts extracted from online sources about each concept, the SDG-HUB Knowledge Graph will support a sophisticated understanding of the online discourse and social mood around problems and solutions for meeting Austria’s climate and sustainability goals.

We hope this to be the first of many projects for Modul University Vienna’s research centre to build domain-specific and national/regional Knowledge Graphs for different purposes. MODUL Technology can apply the same technologies and methodologies – plus lessons learnt in SDG-HUB – to support its future knowledge extraction and modelling work.

MODUL Technology : https://www.modultech.eu

SDG-HUB project: http://www.sdghub.at

Contact: Asst.-Prof. Dr Lyndon Nixon, nixon@modultech.eu  

The EU’s Horizon Europe programme funding research and innovation is coming to an end, and one of the last Horizon Europe projects supporting the media industry also recently came to a successful conclusion. ReTV stood for “re-inventing television for the interactive age”. With dropping viewing time for traditional broadcast TV and an audience shifting to online streaming, Europe’s media industry needs to adapt to distributing their videos in multiple digital channels like Instagram and TikTok and needs tools to help them publish the right video at the right time to get the attention of an online audience overwhelmed by content availability.

It was led by MODUL Technology – the research spin-out of MODUL University Vienna where faculty and researchers collaborate to innovate in new media technologies. A cross-European consortium developed online services and software that help media organizations such as broadcasters to identify the topics of heightened interest to their audiences, find media assets in their collections related to those topics, adapt media content for publication on digital channels such as social networks and schedule the optimal time to publish. MODUL brought data science and AI innovation to the project work, backed by its access to a huge scale data collection and annotation pipeline which provides millions of Webpage texts and social media posts for analysis.

Firstly, we have developed a large knowledge base of global and local events. Media organizations benefit from knowing about future events relevant to their audiences, and we can configure event listings to the interests of each audience. Secondly, we provide advanced components for data analysis that can extract the most significant keywords, topics and stories from the stream of online content. Thirdly, we combine our knowledge of events and topics over time with the latest techniques in machine learning and deep learning to provide topic prediction capabilities, i.e. suggest the topics that will be popular with the audience at a future date. All of our knowledge modelling, knowledge extraction and predictive analytics capabilities continue to be used in other ongoing projects in diverse areas such as brand communication, mobility and digital marketing.

Finally, MODUL Technology demonstrated its capacity to professionally co-ordinate a large EU project through to its successful conclusion. As is written in the project’s final review report, a group of experts evaluated the results of ReTV and stated: “The project has delivered exceptional results with significant immediate or potential impact on relevant stakeholders, namely, content producers, distributors, and broadcasters. The results can bring innovation and competitiveness advantage to those stakeholders by providing them the ability to identify trends and relevant content in a semi-automated or assisted manner, consequently producing and publishing content through the best channels and meeting users’ expectations.“

While we are proud of the results of ReTV and the successful co-ordination by MODUL Technology, now it is time for the team behind this research spin-out of MODUL University Vienna to continue to innovate in data science and AI, and look forward to new challenges funded by the soon-to-be-launched EU Horizon Europe funding program.

Contact for ReTV and MODUL Technology: Asst.Prof. Dr. Lyndon Nixon, nixon AT modultech DOT eu

A key concern of TV providers is to exploit the rapidly expanding digital possibilities for networking and publishing quickly and accurately. Now international experts, with significant contributions from Austrian partners MODUL Technology and webLyzard, are beginning to address this problem. The EU project ReTV, funded to the tune of 3.5 million euros, will provide TV providers with a basis for decision-making with a view to adapting existing content to today’s enormous range of digital networks quickly and efficiently.                                                      

Leading media technology experts across Europe have joined forces to enable TV providers to be more agile and responsive to increasing competition from new digital media. Their declared goal is to develop a “trans-vector platform” that provides TV providers with fast, reliable information on who consumes their content as well as when, where and how they do so. It will enable providers to make sound decision-making about future publication of content on those social networks and digital distribution channels as well as where content adaptation is likely to pay off.

Targeted Adaptation

Dr. Lyndon Nixon, CTO of MODUL Technology and assistant professor at the Institute for New Media Technology at MODUL University Vienna comments: “TV providers have to distribute their content through multiple channels such as social media, mobile apps, hybrid TV and digital archives. But compared to print media – which face similar pressures – their content is technically much more complex. Deciding which content should be adapted and in what way is therefore essential for meeting the demands of consumers in a cost-effective manner.”

This is where the ReTV project comes in. The project is coordinated by MODUL Technology, collaborating with webLyzard and other partners from Germany, Switzerland, Greece and the Netherlands. The cooperative project is divided into three clearly defined sections, the results of which will be of enormous value to TV providers. In addition to “aggregation”, i.e. the establishment of a steadily growing directory of TV content, “analysis” and “adaptation” of such content are key elements of the project.

Aggregation & Annotation

More than 10,000 hours of video content and over 50 million documents will be collected and processed from news sources, social media and TV station websites every month. This huge volume of data will then be automatically analysed, and relevant metadata will be appended to every document. Besides “hard” facts, such as links, names and salient visual features, the metadata contain an automatic evaluation of online mood regarding the topics, persons or organizations mentioned in the content. 

Analysis & Adaptation

Professor Arno Scharl, Managing Director of collaborating partner webLyzard technology, describes how ReTV works: “In the analytical stage, the webLyzard platform is used to capture content trends in social media. This allows intelligent recommendations to be made regarding the adaptation of existing content and the focus for new productions. In this way ReTV will help to optimise advertising strategies, for example by referencing socially relevant topics that are currently being actively discussed by consumers.”

In addition, ReTV will make forecasts about the optimal timing of the release and expected success of original and adapted content. Thanks to the continuous collection and processing of relevant data from a wide range of different sources, the system is also able to learn. If the actual success achieved deviates from predictions, the system will automatically be optimised. In the coming years, ReTV will thus strengthen the competitive situation of European media companies in today’s networked, global market for video content.

In January 2016, MODUL Technology GmbH welcomed its first new year 2016 its first self-acquired EU research project: In Video Veritas (InVID)!

InVID sets out to solve a very difficult problem: verifying the truthfulness of video content posted on social networks which claim to show news events. News agencies have to repeatedly deal with fake, manipulated or misrepresented videos being spread online in the aftermath of a news story. To retain their viewers’ trust, they want to be sure of the authenticity of such video before using it in their news reporting. InVID will provide a complete toolset, driven by various innovative technologies, to aid journalists and newsrooms in semi-automatically determining if a video is trustworthy or not.

MODUL Technology GmbH will develop in this project the social media data ingestion pipeline. Data ingestion will be driven by in-time news event detection, since at any one time the InVID platform should only be collecting relevant candidate media for the verification process, i.e. videos being posted which claim to show something related to a current news event. Detected events will be appropriately labelled so that a query mechanism which is regularly updated can retrieve candidate media items from the social platforms. These items, in order to allow for user friendly search and browsing in the applications which use the platform data, will be richly annotated according to their content and structure, with a focus on unambiguous entity detection (e.g. for determining the location claimed to be shown in the video). This pipeline will be integrated as part of the InVID project into the webLyzard Web Intelligence Platform, which forms the core of the InVID platform solution.

MODUL Technology’s news event detection and dynamic social media querying components can help drive any organisations data ingestion needs, e.g. for online media monitoring as part of a Marketing Intelligence approach or for social media browsing as part of an UGC re-use strategy on a Website or a social network channel. Let us work it out for you, contact us now!