BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20230124T171525Z
LOCATION:C141
DTSTART;TZID=America/Chicago:20221113T083500
DTEND;TZID=America/Chicago:20221113T093500
UID:submissions.supercomputing.org_SC22_sess421_misc168@linklings.com
SUMMARY:Dealing With Unmanageable Data Sizes in Cosmology
DESCRIPTION:Workshop\n\nDealing With Unmanageable Data Sizes in Cosmology\
 n\nLukic\n\nModern sky surveys conducted by powerful telescopes are some o
 f the largest data generators in science, quite popular and visible to the
  broader audience.  However, what goes largely unnoticed is the fact that 
 cosmological simulations often have to produce even larger data sets in or
 der to scientifically interpret these observations: we need many models em
 ploying different plausible physics, as well as to ensure that statistical
  errors of our predictions are less than observational errors.  To maintai
 n desired accuracy, modern simulations track the time evolution of trillio
 ns of elements over thousands of timesteps.  For such large runs, storing 
 a large number of time steps for later analysis is not a viable strategy a
 nymore, and beyond-exascale forecasts point to growth in flops continually
  outpacing growth of disk space as well as network bandwidth, making the p
 ost processing strategy increasingly impossible.  In this talk, I will go 
 over the difficulties we are facing with large data sizes and which presen
 t major technological roadblock.  Then I will present some of our existing
  lines of attack on this problem, including different compression methods,
  surrogate modeling, as well as our design for running multiple codes in s
 itu: using coroutines and position independent executables we enable coope
 rative multitasking between simulation and analysis, allowing the same exe
 cutables to post-process simulation output, as well as to process it on th
 e fly, both in situ and in transit.\n\nSession Format: Recorded\n\nRegistr
 ation Category: Workshop Reg Pass
END:VEVENT
END:VCALENDAR