Dataset Identification:
Resource Abstract:
- People in the locality of earthquakes are publishing anecdotal information about the shaking within seconds of their occurrences
via social network technologies, such as Twitter. In contrast, depending on the size and location of the earthquake, scientific
alerts can take between two to twenty minutes to publish. The goals of this project are to assess earthquake damage and effects
information, as impacts unfold, by leveraging expeditious, free and ubiquitous social-media data to enhance our response to
earthquake damage and effects. <br /> <br /> The project leverages an existing system that performs data acquisition
from Twitter of earthquake related tweets, and geocoding from Yahoo. The acquired data stream is archived into a Postgres
database, filtered and monitored by the detection application (Tedect). The social media data acquisition and distribution
application (TED) was enhanced to concurrently feed the focused data stream into an Elasticsearch index. Elasticsearch is
an open source tool built on Apache Lucene, a high-performance text search engine library, that creates optimized searchable
data in JSON format that is in turn visualized by Kibana, a web-based interactive interface. Elasticsearch and Kibana are
new technologies that the team was not aware of at the time the proposal was written, but are proving to be a valuable, low
cost way of providing rapid indications of earthquake significance and impacts. <br /> <br /> As a proof of concept
the geographic data associated with the project was upgraded to make it more readily and programmatically available for collaborators.
The creation of GIS services for sharing and distributing data was deemed the easiest data feed source to most readily integrate
with a variety of GIS end users, including FEMA. Implementation of two service types on a per event basis were established
for manual configuration following significant, detected events. The first is a point based tweet dataset that can be
mapped and queried. The second is a raster based heat map of tweet density which covers the region of interest.
Once manually configured these services are created to acquire related data at three distinct time intervals after an event,
which supports the series of post detection summary reports. <br /> <br /> All of this data acquisition, processing,
analysis, and organization leads to the automated production of valuable earthquake significance, damage and effects information
for use by practitioners (scientists, earthquake responders and collaborators). The rapid availability of such earthquake
characterization information allows practitioners to customize their alerts, from this system, according to event significance.
<br /> <br /> The proposed project included integration of additional social media data sources. However, further analysis,
of data from Flickr and Instagram, found the signal to noise ratio too high to be of benefit in the scope of this project.
Future efforts and tool development to more efficiently filter and assess the data may prove fruitful, however, with the resources
available focusing on Twitter data and media proved more effective. <br /> <br /> The outcomes from this project support
USGS goals of advancing our understanding of earthquake effects, establishing data sharing and collaboration, as well as providing
additional rapid situational awareness. <br /> <br /> Deliverables <ul> <li>Updated Social Media Earthquake Acquisition
and Distribution Application, also known as TED. The enhancements to the system include derivation and distribution of earthquake
significance summary reports and archival of tweet based event detections. The most recent code is available on the USGS stash
git repository at: https://my.usgs.gov/stash/projects/NEIC/repos/ted/browse. Efforts are still underway to migrate to publically
accessible git repository, which is planned for completion June 2015.</li> <li>Updated Social Media Earthquake Detection Application,
also known as Tedect, was updated to send tweet based event detections into TED and performance improved. The most recent
code is on the USGS stash git repository at: https://my.usgs.gov/stash/projects/NEIC/repos/tedect/browse. An older reviewed
version is publically available at https://github.com/mguy-usgs/tedect.</li> <li>Earthquake Significance Summary Reports are
automatically derived for tweet based event detections at five and ten minutes after events occur. The reports give rapid
indications of public interest in the event, for use internally and by collaborators. </li> <li>Web Services <ul> <li>The
team established a workflow for using the Kibana visual data analysis tools to, in near real time, to determine earthquake
significance as well as impacts and effects, as illustrated in Figure 2 and 3. This open source service currently has no free
mechanism to secure the data in order to preserve data integrity, so it remains internal.</li> <li>As a proof of concept,
with coordination with FEMA, a GIS service feed for detected events was implemented. An example heatmap created from this
service included in figure 4 and is available at: http://geohazards.usgs.gov/arcgis/rest/services/NapaHeatmap/MapServer.</li>
</ul> </li> <li>Integrated Social Media and Seismic Earthquake Dataset - this integrated dataset provides validation of the
system, and analysis demonstrates the system performance as described in figures 5, 6 and 7. The analysis of this dataset
was shared publically at scientific conferences including American Geophysical Union (AGU) in December 2014 (https://agu.confex.com/agu/fm14/meetingapp.cgi#Paper/13007)
and Association of Computing Machinery (ACM) Special Interest Group Knowledge Discovery and Data Mining (SIGKDD) Learning
about Emergencies from Social Informaion (LESI) workshop in August 2014 (https://sites.google.com/site/kddlesi2014/program/papers).</li>
</ul> <br /> Note: <em>this description is from the CDI FY14 Annual Report</em>
Citation
- Title Characterization of Earthquake Damage and Effects Using Social Media Data
-
- Award Date Date
2014
Topic Category:
geoscientificInformation
Label
Keywords
Applications
Web Services
Semantics
Tools
Data
2014
Theme keywords (theme):
thesaurus name >
- Title Activity > Observation
-
- publication Date
2017-01-18
other Citation Details
Cinergi keyword enhanced at Wed Jan 18 02:48:11 UTC 2017
Theme keywords (theme):
thesaurus name >
- Title Resource Type > Web Service
-
- publication Date
2017-01-18
other Citation Details
Cinergi keyword enhanced at Wed Jan 18 02:48:11 UTC 2017
Theme keywords (theme):
thesaurus name >
- Title Process > Geologic Processes
-
- publication Date
2017-01-18
other Citation Details
Cinergi keyword enhanced at Wed Jan 18 02:48:11 UTC 2017
Theme keywords (theme):
thesaurus name >
- Title Organization > GOVERNMENT AGENCIES-U.S. FEDERAL AGENCIES
-
- publication Date
2017-01-18
other Citation Details
Cinergi keyword enhanced at Wed Jan 18 02:48:11 UTC 2017
Theme keywords (theme):
thesaurus name >
- Title Organization > NON-GOVERNMENT/NON-PROFIT ORGANIZATIONS
-
- publication Date
2017-01-18
other Citation Details
Cinergi keyword enhanced at Wed Jan 18 02:48:11 UTC 2017
Theme keywords (theme):
thesaurus name >
- Title Feature > Hydrologic Feature
-
- publication Date
2017-01-18
other Citation Details
Cinergi keyword enhanced at Wed Jan 18 02:48:11 UTC 2017
Theme keywords (theme):
thesaurus name >
- Title Resource Type > Repository
-
- publication Date
2017-01-18
other Citation Details
Cinergi keyword enhanced at Wed Jan 18 02:48:11 UTC 2017
Theme keywords (theme):
thesaurus name >
- Title Method > Information Processing
-
- publication Date
2017-01-18
other Citation Details
Cinergi keyword enhanced at Wed Jan 18 02:48:11 UTC 2017
Theme keywords (theme):
thesaurus name >
- Title Property > Measure
-
- publication Date
2017-01-18
other Citation Details
Cinergi keyword enhanced at Wed Jan 18 02:48:11 UTC 2017
Theme keywords (theme):
thesaurus name >
- Title Equipment > Instrument
-
- publication Date
2017-01-18
other Citation Details
Cinergi keyword enhanced at Wed Jan 18 02:48:11 UTC 2017
Theme keywords (theme):
thesaurus name >
- Title Science Domain > Earth Science
-
- publication Date
2017-01-18
other Citation Details
Cinergi keyword enhanced at Wed Jan 18 02:48:11 UTC 2017
Theme keywords (theme):
thesaurus name >
- Title Science Domain > Computer Science
-
- publication Date
2017-01-18
other Citation Details
Cinergi keyword enhanced at Wed Jan 18 02:48:11 UTC 2017
Theme keywords (theme):
thesaurus name >
- Title Resource Type > Data Set
-
- publication Date
2017-01-18
other Citation Details
Cinergi keyword enhanced at Wed Jan 18 02:48:11 UTC 2017
purpose:
Develop process to characterize earthquake damage with image data for research/emergency response scalable to other hazards
(floods)
Resource language:
eng
Back to top:
Metadata data stamp:
2015-06-17T19:23:50Z
Metadata Constraints
-
- Constraints
-
- Use limitation statement:
- none
Metadata contact
-
principalInvestigator
- individual Name Michelle Guy
- position Name Computer Scientist
-
- Contact information
-
-
- Telephone
-
- Fax 3032738450
-
- Address
-
- 1711 Illinois Street, Golden, CO, 80401-1435
Country US
- electronic Mail Address mguy@usgs.gov
Linkage for online resource
- URL: https://www.sciencebase.gov/directory/person/4043
Metadata contact
-
pointOfContact
- individual Name Scott R Horvath
- position Name Web and Social Media Chief
-
- Contact information
-
-
- Telephone
-
- Voice 7036484011
- Fax 7036484466
-
- Address
-
- 12201 Sunrise Valley Dr, Reston, VA, 20192-0002
Country US
- electronic Mail Address shorvath@usgs.gov
Linkage for online resource
- URL: https://www.sciencebase.gov/directory/person/4731
Metadata contact
-
pointOfContact
- individual Name Douglas Bausch
-
- Contact information
-
-
- Address
-
- electronic Mail Address Douglas.Bausch@dhs.gov
-
- Linkage for online resource
-
- URL: https://www.sciencebase.gov/directory/person/12991
Metadata contact
-
pointOfContact
- individual Name Gregory M Smoczyk
- position Name IT Specialist
-
- Contact information
-
-
- Telephone
-
- Voice 3032738625
- Fax 3032738600
-
- Address
-
- 1711 Illinois Street, Golden, CO, 80401-1435
Country US
- electronic Mail Address gsmoczyk@usgs.gov
Linkage for online resource
- URL: https://www.sciencebase.gov/directory/person/9676
Metadata contact
-
principalInvestigator
- individual Name Paul S Earle
- position Name Supervisory Development Geophysicist
-
- Contact information
-
-
- Telephone
-
- Voice 3032738417
- Fax 3032738450
-
- Address
-
- 1711 Illinois Street, Golden, CO, 80401-1435
Country US
- electronic Mail Address pearle@usgs.gov
Linkage for online resource
- URL: https://www.sciencebase.gov/directory/person/2807
Metadata scope code
dataset
Metadata language
eng USA
Metadata character set encoding:
utf8
Metadata standard for this record:
ISO19115
standard version:
1.0
Metadata record identifier:
53207404e4b0224be0a979e1
URI for dataset described:
> https://www.sciencebase.gov/catalog/item/53207404e4b0224be0a979e1
Metadata record format is ISO19139-2 XML (MI_Metadata)