First workshop on:

Exploitation of Social Media for Emergency Relief and Preparedness (SMERP)

9th April, 2017


Co-located with:

ECIR 2017, 8th - 13th April, 2017,
Aberdeen, Scotland UK

HTML5 Icon Venue: Sir Ian Wood Building, Robert Gordon University campus, Garthdee Road, Aberdeen, AB10 7GJ.

Submit to this Workshop!

Updates

Call for Participation

The enormous growth of the use of social media during emergency situations has led to increasing amounts of information being available to assist emergency relief operations. Additionally, Online Social Media (OSM) content can also be utilised for emergency preparedness and early warning systems. Effective exploitation of the crowdsourced content posted on OSM requires reliable real-time IR methodologies, and integration of OSM content with other information sources. This workshop will explore the multifarious aspects of effective information extraction and exploitation from social media, for emergency relief, as well as emergency preparedness. Along with a peer-reviewed research paper track, the workshop will include a TREC-style data challenge. The two tracks (both overview and details) are as given below:


Peer-review Track

This track of the workshop aims to provide a research platform to explore the role of social media in emergency relief and management and requests for original research contributions related to the theme Exploitation of Social Media for Emergency Relief and Preparedness. The detailed aims and scope of this track are provided here.

Data Challenge Track

This track of the workshop aims to promote development of IR methodologies for some practical challenges that need to be addressed during an emergency event, along with thorough evaluation and comparison of the methodologies. To fulfil this objective, we present a data challenge, following the style of TREC tracks. We will provide a large dataset of microblogs posted during the earthquake in Italy in August 2016, along with a set of queries. The participants are invite to participate in the following challenges:

  • Text retrieval
  • Text summarization

The participants will be invited to submit solutions to the said challenges. We will arrange for evaluation of the submitted results. The top performing teams who participate in the data challenge will also be invited for participation in the workshop. More details are given here.    

Peer-review Track (details)

Aims and Scope

This track requests for original contributions related to the theme, which includes (but not limited to):

  • Information retrieval and extraction from short, noisy content posted on OSM.
  • Applications of data mining, NLP and machine learning for processing OSM content.
  • Aggregating information from multiple OSM and online / offline resources.
  • Addressing the code-mixed and informal vocabulary of OSM content.
  • Detection of events and emerging themes.
  • Real-time management and summarization of dynamic content streams.
  • Detection of rumours, and identification of trustworthy sources and information.
  • Geo-tagging and geo-localisation of content and sources.
  • Social network models for information diffusion in emergency situations.
  • Identifying disaster-prone or accident-prone regions and infrastructures.
  • Crowdsourcing systems for emergency preparedness and disaster relief.
  • Mining interactions among emergency preparedness and relief groups.

Submission

All submissions must be written in English following the LNCS author guidelines. Full papers must not exceed 10 pages, and short papers must not exceed 6 pages, including all diagrams, references, and appendices. Submitted papers should include the names and affiliations of all authors.

Papers (PDF only) should be submitted electronically through Easychair.

Submitted papers will be peer-reviewed, and the accepted papers will be included in the workshop proceedings, which will be published online via the CEUR workshop proceedings publication service (ceur-ws.org, ISSN 1613-0073).

At least one authors of each accepted paper is expected to attend the workshop and present the paper.

Authors of some of the accepted papers will be invited to submit extended versions to a Special Issue on “Exploitation of Social Media for Emergency Relief and Preparedness” of the journal Information Systems Frontiers.

Important dates for Peer-review Track

Data Challenge Track (details)

The motivation of the data challenge is to promote development of IR methodologies that can be used to extract important information from social media during emergency events, and to arrange for comparative evaluation of the methodologies. There will be two sub-tracks / challenges:
  • Text Retrieval
  • Text Summarization
Participants can choose to participate in either or both of the sub-tracks.

Dataset

Tweet collection: A large dataset of microblogs (tweets) posted on Twitter during the earthquake in Italy in August 2016, will be provided. Since the Twitter terms do not allow public sharing of tweets, only the tweetids of the tweets will be provided, along with a Python script that can be used to download the tweets using the Twitter API.
Topics: A set of topics (information needs) will be provided. Each topic will identify a broad information need during a disaster, such as – what resources are needed in the disaster­ affected area, what resources are available, what damages are being reported, etc. Specifically, each topic will be in TREC format, and will contain a title, a brief description, and a more detailed narrative on what type of tweets will be considered relevant to the topic.

Text Retrieval sub-track

In this sub-track, the participants are required to develop methodologies for extracting tweets that are relevant to each topic, with high precision (i.e., ideally, only the relevant tweets should be identified) as well as high recall (i.e., ideally, all relevant tweets should be identified).
This sub-track will have two levels:

Level 1

In this level, the tweets collected during the first day (24 hours) after the earthquake will be provided, and the participants will be asked to extract tweets relevant to each specified topic. The extracted tweets should be submitted to us, and we will arrange for evaluation of some of the top-ranked results of each submission.

Level 2

In this level, the tweets collected during the second day (24 hours) after the earthquake will be provided. Additionally, some of the tweets that are actually relevant to each topic, from among the tweets used in Level 1, will be provided. The participants will again be expected to retrieve tweets relevant to each topic from among the tweets posted during the second day.
The purpose of the second level is to provide the participants some idea about which tweets are considered relevant, so that they can apply this knowledge to improve retrieval on an unseen set of tweets. Thus, this exercise will present a platform to evaluate the dynamic reusability and adaptability of the IR system in a practical scenario.

Evaluation

The retrieval results will be evaluated by Mean Average Precision (MAP) and Recall.


Text Summarization sub-track

In this sub-track, the participants are required to develop methodologies for summarizing the information that is relevant to each topic. Specifically, summaries of length at most 300 words should be generated for each topic. Both extractive and abstractive summarization methods are welcome.
This sub-track will also have two levels:

Level 1

In this level, the tweets collected during the first day (24 hours) after the earthquake will be provided, and the participants will be required to submit a summary (of maximum 300 words) for each topic. We will arrange for evaluation of the summaries.


Level 2

In this level, the tweets collected during the second day (24 hours) after the earthquake will be provided. Additionally, we will indicate which parts of the summaries produced from the tweets of the first day, are important for the second day. The participants will again be required to submit summaries produced from the tweets of the second day. It will be expected that the parts that were indicated to be important after the first day, will be given more importance while generating the summaries for the second day.
The purpose of the second level is to provide the participants some idea of which types of information are important for each topic after the first day, so that they can tune the algorithms while generating the summaries on the tweets of the second day. Thus, this exercise will present a platform to evaluate the dynamic adaptability of the summarization system in a practical scenario.

Evaluation

The Text Summarization results will be evaluated in two ways:
  1. Using automatic measures like ROUGE score, BLEU score, etc.
  2. Using manual assessment of the quality of the summaries

Note

  • A participating team should contain between 1 and 5 members.
  • For both the sub-tracks, a team can participate in Level 1 only, or in both the levels. However, a team can not participate in Level 2 only (i.e., to participate in Level 2, participation in Level 1 is mandatory).
  • The two teams with the best performances in each sub-track will be invited to submit papers based on their methodologies, which will be included in the workshop proceedings (through a process of shepherding).
  • Workshop proceedings will be archived online in the CEUR workshop proceedings publication service (http://ceur-ws.org/, ISSN 1613-0073).

Communication

Interested participants should send email to smerp2017 [at] gmail [dot] com, stating the names and affiliations of the team members, and which sub-track(s) they wish to participate in (Text Retrieval or Text Summarisation or both). The organizers will send the dataset by replying to the mail. For both sub-tracks, there should be separate data requests for Levels 1 and 2. All further correspondences, viz., result submission, evaluation result declaration and request for paper submission, will be done via the same mail id.

Important dates for Data Challenge Track

Results of Data Challenge Track : Task 1 (Extraction) and Task 2 (Summarization)

Organizing Committee

Marie-Francine Moens, KU Leuven, Belgium
Gareth Jones, Dublin City University, Ireland
Saptarshi Ghosh, IIT Kharagpur, India; IIEST Shibpur, India
Debasis Ganguly, IBM Research Labs, Dublin, Ireland
Tanmoy Chakraborty, University of Maryland, College Park, USA
Kripabandhu Ghosh, IIT Kanpur, India; ISI Kolkata, India

Technical Program Committee

Haithem Afli, Adapt Centre, School of computing, Dublin City University, Ireland
Arnab Bhattacharya, IIT Kanpur, India
Pushpak Bhattacharyya, IIT Bombay, India
Joydeep Chandra, IIT Patna, India
Monojit Choudhury, Microsoft Research, India
Ravindranath Chowdary C, IIT (BHU), India
Amitava Das, IIIT Sri City, India
Niloy Ganguly, IIT Kharagpur, India
Parantapa Goswami, Université Joseph Fourier, France
Pawan Goyal, IIT Kharagpur, India
Roman Klinger, Stuttgart University, Germany
Bivas Mitra, IIT Kharagpur, India
Animesh Mukherjee, IIT Kharagpur, India
Philippe Muller, Paul Sabatier University, France
Jiaul Hoque Paik, IIT Kharagpur, India
Sukomal Pal, IIT (BHU), India
Alexander Panchenko, TU Darmstadt, Germany
Rishiraj Saharoy, Max Planck Institute for Informatics, Germany
Anil Kr. Singh, IIT (BHU), India
V. G. Vinod Vydiswaran, University of Michigan, USA
Joachim Wagner, Dublin City University, Ireland

Submissions

The papers for the Peer-review Track should be submitted via Easychair.

The correspondence for the Data Challenge Track will be done via smerp2017 [at] gmail [dot] com.

For more details please see Call for Participation.

Program

09:00 - 09:10 - Opening
09.10 - 10.10 - Keynote Address by Muhammad Imran: Time-Critical Analysis of Evolving Social Media Streams During Sudden-Onset Events [pdf]
10.10 - 10.25 - [Accepted paper] Situational Awareness for Low Resource Languages: the LORELEI Situation Frame Annotation Task. S. Strassel, A. Bies, J. Tracey [pdf]
10.25 - 10.40 - [Accepted paper] Weakly Supervised Classification of Tweets for Disaster Management. G. K. Palshikar, M. Apte, D. Pandita [pdf]
10.40 - 11.00 - Tea / Coffee Break
11.00 - 11.20 - Reporting on SMERP Data Challenge [Speaker: S. Ghosh]
11.20 - 11.40 - Presentations by teams participating in data challenge:
(i) Team from Radboud University [Speaker: H. Ali] [pdf]
(ii) Team from Dublin City University [Speaker: G. Jones] [pdf]
11.40 - 11.55 - [Accepted paper] Identification of disaster-implicated named entities. D. M. Nemeskey, I. Benyeda, A. Kornai [pdf]
11.55 - 12.10 - [Accepted paper] Microblog Retrieval in a Disaster Situation: A New Test Collection for Evaluation. M. Basu, A. Roy, K. Ghosh, S. Bandyopadhyay, S. Ghosh [pdf]
12.10 - 12.30 - Discussion - I: Designing effective data challenges on exploiting social media for emergency relief and preparedness - the questions
12:30 - 13:30 - Lunch break
13:30 - 14:30 - Keynote Address by Richard McCreadie: Summarizing the Situation with Social Media Streams [pdf]
14:30 - 14:45 - [Accepted paper] Microblog Retrieval During Disasters: How To Create Ground Truths? R. Soni, S. Pal [pdf]
14:45 - 15:15 - Discussion - II: Designing effective data challenges on exploiting social media for emergency relief and preparedness - the answers
15:15 - 15:35 - Tea / Coffee Break
15:35 - 16:50 - Panel discussion: Where to go from here? The open challenges in exploiting social media for disaster relief and preparedness.
16:50 - 17:00 - Closing

The names of speakers are in bold, titles of talks italicized.

Important Dates


Peer-review Track

January 27, 2017 Paper submission due
March 6, 2017 Notification of acceptance
March 15, 2017 Camera-ready submission due
April 09, 2017 Workshop

Data Challenge Track

November 07, 2016 Data for Level 1 released
December 20, 2016 Submission of results and methodology description for Level 1
December 31, 2016 Level 1 evaluation results declared; Level 2 data released
February 6, 2017 Submission of results and methodology description for Level 2
February 22, 2017 Level 2 evaluation results declared
March 7, 2017 Submission of papers by top two teams for each track
March 08, 2017 Notification of reviews after shepherding
March 15, 2017 Camera-ready submission due
April 09, 2017 Workshop

All these dates follow the Anywhere on Earth time zone.

Registration

Please visit the ECIR 2017 Registration page for details.

Contacts

For queries about the data challenge, you can mail smerp2017 [at] gmail [dot] com.

For any other queries about the workshop, you can mail the following organizers:
Saptarshi Ghosh: saptarshi [dot] ghosh [at] gmail [dot] com
Kripabandhu Ghosh: kripa [dot] ghosh [at] gmail [dot] com