Opendata, web and dolomites

MultiMT SIGNED

Multi-modal Context Modelling for Machine Translation

Total Cost €

0

EC-Contrib. €

0

Partnership

0

Views

0

 MultiMT project word cloud

Explore the words cloud of the MultiMT project. It provides you a very rough idea of what is the project "MultiMT" about.

examples    cues    incorrect    economic    drastically    social    beds    claims    global    dominant    applies    media    reviews    types    miss    models    visual    useless    significantly    algorithms    paradigm    contextual    textual    contain    pair    vision    performing    little    documents    devise    idea    source    context    datasets    learns    metadata    acquire    disruptive    statistical    translate    multilingual    automatically    computer    date    translation    mt    critical    ways    human    images    positive    translations    natural    regardless    form    breakthrough    representations    interdisciplinary    barriers    machine    learning    translators    smt    inference    twitter    actual    standard    surrounding    modal    texts    appropriate    disregard    segment    author    comprehension    enormous    draws    lower    expertise    content    reading    posts    document    nlp    convey    completely    language    reference    translated    communication    sought    translating   

Project "MultiMT" data sheet

The following table provides information about the project.

Coordinator
IMPERIAL COLLEGE OF SCIENCE TECHNOLOGY AND MEDICINE 

Organization address
address: SOUTH KENSINGTON CAMPUS EXHIBITION ROAD
city: LONDON
postcode: SW7 2AZ
website: http://www.imperial.ac.uk/

contact info
title: n.a.
name: n.a.
surname: n.a.
function: n.a.
email: n.a.
telephone: n.a.
fax: n.a.

 Coordinator Country United Kingdom [UK]
 Total cost 1˙493˙771 €
 EC max contribution 1˙493˙771 € (100%)
 Programme 1. H2020-EU.1.1. (EXCELLENT SCIENCE - European Research Council (ERC))
 Code Call ERC-2015-STG
 Funding Scheme ERC-STG
 Starting year 2016
 Duration (year-month-day) from 2016-07-01   to  2021-06-30

 Partnership

Take a look of project's partnership.

# participants  country  role  EC contrib. [€] 
1    IMPERIAL COLLEGE OF SCIENCE TECHNOLOGY AND MEDICINE UK (LONDON) coordinator 1˙010˙513.00
2    THE UNIVERSITY OF SHEFFIELD UK (SHEFFIELD) participant 483˙257.00

Map

 Project objective

Automatically translating human language has been a long sought-after goal in the field of Natural Language Processing (NLP). Machine Translation (MT) can significantly lower communication barriers, with enormous potential for positive social and economic impact. The dominant paradigm is Statistical Machine Translation (SMT), which learns to translate from human-translated examples.

Human translators have access to a number of contextual cues beyond the actual segment to translate when performing translation, for example images associated with the text and related documents. SMT systems, however, completely disregard any form of non-textual context and make little or no reference to wider surrounding textual content. This results in translations that miss relevant information or convey incorrect meaning. Such issues drastically affect reading comprehension and may make translations useless. This is especially critical for user-generated content such as social media posts -- which are often short and contain non-standard language -- but applies to a wide range of text types.

The novel and ambitious idea in this proposal is to devise methods and algorithms to exploit global multi-modal information for context modelling in SMT. This will require a significantly disruptive approach with new ways to acquire multilingual multi-modal representations, and new machine learning and inference algorithms that can process rich context models. The focus will be on three context types: global textual content from the document and related texts, visual cues from images and metadata including topic, date, author, source. As test beds, two challenging user-generated datasets will be used: Twitter posts and product reviews.

This highly interdisciplinary research proposal draws expertise from NLP, Computer Vision and Machine Learning and claims that appropriate modelling of multi-modal context is key to achieve a new breakthrough in SMT, regardless of language pair and text type.

 Publications

year authors and title journal last update
List of publications.
2018 Madhyastha, Pranava; Wang, Josiah; Specia, Lucia
End-to-end Image Captioning Exploits Multimodal Distributional Similarity
published pages: , ISSN: , DOI:
Proceedings of the Bristish Machine Vision Conference (BMVC) 1 2019-10-09
2018 Wang, Josiah; Madhyastha, Pranava; Specia, Lucia
Object Counts! Bringing Explicit Detections Back into Image Captioning
published pages: , ISSN: , DOI:
Proceedings of 2018 Conference of the North American Chapter of the Association for Computational Linguistics 2 2019-10-08
2018 Madhyastha, Pranava; Wang, Josiah; Specia, Lucia
Defoiling Foiled Image Captions
published pages: , ISSN: , DOI:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2018) 1 2019-10-08
2017 Elliott, Desmond; Frank, Stella; Barrault, Loïc; Bougares, Fethi; Specia, Lucia
Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Description
published pages: 215--233, ISSN: , DOI:
Proceedings of the Second Conference on Machine Translation 3 2019-10-08
2017 Salil Deena, Raymond Ng, Pranava Madhyastha, Lucia Specia, and Thomas Hain
Exploring the use of acoustic embeddings in neural machine translation
published pages: , ISSN: , DOI:
IEEE ASRU Workshop 2019-10-08
2017 Salil Deena, Raymond W. M. Ng, Pranava Madhyastha, Lucia Specia, and Thomas Hain
Semi-supervised Adaptation of RNNLMs by Fine-tuning with Domain-specific Auxiliary Features
published pages: 2715–2719, ISSN: , DOI:
INTERSPEECH 2019-10-08
2016 Kashif Shah, Josiah Wang and Lucia Specia
SHEF-Multimodal: Grounding Machine Translation on Images
published pages: , ISSN: , DOI:
Conference on Machine Translation 2019-10-08
2016 Lucia Specia, Stella Frank, Khalil Sima\'an, and Desmond Elliott
A Shared Task on Multimodal Machine Translation and Crosslingual Image Description
published pages: 540–550, ISSN: , DOI:
First Conference on Machine Translation 2019-10-08
2016 Desmond Elliott, Stella Frank, Khalil Sima\'an, and Lucia Specia
Multi30K: Multilingual English-German Image Descriptions
published pages: 70–74, ISSN: , DOI:
5th Workshop on Vision and Language 2019-10-08
2018 C. Lala, P. Swaroop Madhyastha, Carolina Scarton, L. Specia
Sheffield submissions for wmt18 multimodal translation shared task
published pages: , ISSN: , DOI:
Conference on Machine Translation 2019-10-08
2018 PRANAVA MADHYASTHA, JOSIAH WANG, LUCIA SPECIA
The role of image representations in vision to language tasks
published pages: 415-439, ISSN: 1351-3249, DOI: 10.1017/s1351324918000116
Natural Language Engineering 24/03 2019-10-08
2018 L. Barrault, F. Bougares, L. Specia, C. Lala, D. Elliott, S. Frank
Findings of the Third Shared Task on Multimodal Machine Translation
published pages: , ISSN: , DOI:
Conference on Machine Translation 2019-10-08
2018 STELLA FRANK, DESMOND ELLIOTT, LUCIA SPECIA
Assessing multilingual multimodal image description: Studies of native speaker preferences and translator choices
published pages: 393-413, ISSN: 1351-3249, DOI: 10.1017/s1351324918000074
Natural Language Engineering 24/03 2019-10-08
2017 Pranava Swaroop Madhyastha, Josiah Wang, and Lucia Specia
Sheffield MultiMT: Using Object Posterior Predictions for Multimodal Machine Translation
published pages: 470–476, ISSN: , DOI:
Second Conference on Machine Translation 2019-10-08
2019 O. Caglayan, L. Barrault, P. Madhyastha, L. Specia
Probing the Tole of Images in Multimodal Machine Translation
published pages: , ISSN: , DOI:
NAACL 2019-10-08
2018 R. Sanabria, O. Caglayan, S. Palaskar, D. Elliott, L. Barrault, L. Specia, F. Metze
How2: A Large-scale Dataset for Multimodal Language Understanding
published pages: , ISSN: , DOI:
NeurIPS Workshop on Visually Grounded Interaction and Language (ViGIL) 2019-10-08
2018 Chiraag Lala and Lucia Specia
Multimodal Lexical Translation
published pages: 3810–3817, ISSN: , DOI:
Eleventh International Conference on Language Resources and Evaluation 2019-10-08
2017 Chiraag Lala, Pranava Madhyastha, Josiah Wang, Lucia Specia
Unraveling the Contribution of Image Captioning and Neural Machine Translation for Multimodal Machine Translation
published pages: 197–208, ISSN: 1804-0462, DOI: 10.1515/pralin-2017-0020
The Prague Bulletin of Mathematical Linguistics 108/1 2019-06-19

Are you the coordinator (or a participant) of this project? Plaese send me more information about the "MULTIMT" project.

For instance: the website url (it has not provided by EU-opendata yet), the logo, a more detailed description of the project (in plain text as a rtf file or a word file), some pictures (as picture files, not embedded into any word file), twitter account, linkedin page, etc.

Send me an  email (fabio@fabiodisconzi.com) and I put them in your project's page as son as possible.

Thanks. And then put a link of this page into your project's website.

The information about "MULTIMT" are provided by the European Opendata Portal: CORDIS opendata.

More projects from the same programme (H2020-EU.1.1.)

SWIP (2019)

Extended short-wave infrared pulsed fibre laser

Read More  

ChaperoneRegulome (2020)

ChaperoneRegulome: Understanding cell-type-specificity of chaperone regulation

Read More  

ZARAH (2020)

Women’s labour activism in Eastern Europe and transnationally, from the age of empires to the late 20th century

Read More