🌡️ Protostar Challenge: Climate Risk Intelligence

AberdeenAbileneAikenAkronAlbanyAlbuquerqueAmarilloAshevilleAshlandAstoriaAthensAtlantaAugustaAustinBakersfieldBaltimoreBatonRougeBeckleyBelvidereBirminghamBismarckBluefieldBoiseBostonBuffaloBurlingtonCapeHatterasCedarRapidsCharlestonCharlotteChattanoogaCheyenneChicagoCincinnatiClaytonClevelandCollegeStationColoradoSpringsColumbiaMOColumbiaSCColumbusGAColumbusOHConcordConcordiaCorpusChristiCrescentCityDallasFortWorthDaytonDenverDesMoinesDetroitDodgeCityDubuqueDuluthEUGENEEauClaireElPasoElkoElyErieEscondidoEurekaEvansvilleFargoFlagstaffFortMyersFortSmithFortWayneFresnoGainesvilleGalvestonGlasgowGlensFallsGoodlandGrandForksGrandIslandGrandJunctionGrandRapidsGreatFallsGreenBayGreenvilleSpartanburgGulfportHarrisonHastingsHattiesburgHavreHaywardHelenaHoustonHuntingtonHuntsvilleHuronIndianapolisInternationalFallsJacksonJacksonvilleJeffersonJonesboroKalamazooKansasCityKeyWestKlamathFallsLaCrosseLakeCharlesLanderLansingLewistonLexingtonLincolnLittleRockLosAngelesLouisvilleLynchburgMaconMadisonManchesterMarquetteMcCookMemphisMeridianMiamiMiddletownHarrisburgMiltonMilwaukeeMinneapolisMissoulaMitchellMobileMolineMonroeMontgomeryMorgantownMuscleShoalsMuskegonNashvilleNatchitochesNeedlesNewBrunswickNewYorkNorfolkNENorfolkVANorthPlatteOklahomaOmahaOrlandoPendletonPensacolaPeoriaPhiladelphiaPhoenixPineBluffPittsburghPortlandPortlandMEPuebloPullmanRaleighReddingRenoRichmondRiversideRochesterMNRochesterNYRockfordRoswellRussellvilleSacramentoSaginawSaintCloudSalemSaltLakeCitySanAntonioSanDiegoSaultSteMarieSavannahScottsbluffSeattleShreveportSiouxCitySiouxFallsSouthBendSpencerSpokaneSpringfieldSpringfieldMOStLouisStillwaterTallahasseeTampaToledoTopekaTrentonTriCitiesTucsonTulsaUkiahValentineVancouverWashingtonDCWaterlooWausauWestPalmBeachWheelingWilliamsportWillistonWilmingtonWinnemuccaWorcesterYumaZaneville

Protostar Challenge

Climate Risk Intelligence

AI systems to analyze extreme weather trends and recommend climate adaptation strategies.

Nov 2–13, 2026210 citiesForecast 2019–2025Teams

210 weather stations, 1851–2018 · data: CMU-OSPO/protostar-climate-train-to-2018

1900: +1.2 °F1901: -0.1 °F1902: -0.0 °F1903: -0.8 °F1904: -0.9 °F1905: -0.7 °F1906: +0.2 °F1907: -0.5 °F1908: +0.6 °F1909: -0.2 °F1910: +0.4 °F1911: +0.8 °F1912: -1.3 °F1913: +0.5 °F1914: +0.2 °F1915: -0.1 °F1916: -0.7 °F1917: -1.9 °F1918: +0.3 °F1919: +0.3 °F1920: -0.6 °F1921: +2.4 °F1922: +0.8 °F1923: +0.2 °F1924: -1.1 °F1925: +0.9 °F1926: +0.0 °F1927: +0.7 °F1928: +0.4 °F1929: -0.4 °F1930: +0.8 °F1931: +2.6 °F1932: +0.7 °F1933: +1.7 °F1934: +2.0 °F1935: +0.3 °F1936: +0.5 °F1937: +0.2 °F1938: +2.0 °F1939: +1.8 °F1940: -0.2 °F1941: +1.4 °F1942: +0.4 °F1943: +0.1 °F1944: +0.4 °F1945: +0.1 °F1946: +1.4 °F1947: +0.2 °F1948: +0.0 °F1949: +0.9 °F1950: -0.6 °F1951: -0.6 °F1952: +0.6 °F1953: +1.6 °F1954: +1.2 °F1955: +0.2 °F1956: +0.4 °F1957: +0.4 °F1958: -0.2 °F1959: +0.4 °F1960: -0.7 °F1961: -0.2 °F1962: -0.3 °F1963: -0.1 °F1964: -0.0 °F1965: -0.2 °F1966: -0.6 °F1967: -0.4 °F1968: -0.6 °F1969: -0.6 °F1970: -0.2 °F1971: +0.0 °F1972: -0.7 °F1973: +0.8 °F1974: +0.3 °F1975: +0.1 °F1976: -0.7 °F1977: +0.4 °F1978: -1.0 °F1979: -0.9 °F1980: +0.2 °F1981: +0.7 °F1982: -0.3 °F1983: +0.0 °F1984: +0.1 °F1985: -0.5 °F1986: +1.1 °F1987: +1.5 °F1988: +0.5 °F1989: -0.3 °F1990: +1.9 °F1991: +1.6 °F1992: +0.5 °F1993: -0.3 °F1994: +0.9 °F1995: +0.6 °F1996: -0.7 °F1997: -0.2 °F1998: +2.3 °F1999: +1.5 °F2000: +0.5 °F2001: +1.2 °F2002: +1.0 °F2003: +0.5 °F2004: +0.8 °F2005: +1.4 °F2006: +2.2 °F2007: +1.6 °F2008: +0.2 °F2009: +0.2 °F2010: +1.2 °F2011: +1.5 °F2012: +3.4 °F2013: +0.5 °F2014: +0.2 °F2015: +2.4 °F2016: +3.0 °F2017: +2.6 °F2018: +1.7 °F

1900 → 2018: each stripe is one year, average temperature of the 210 cities vs. 1961–1990

  1. Nov 2Challenge opens
  2. Nov 13Deadline, 11:59 PM ET
  3. After the deadlineFinal Score
  4. ThenWinners

The challenge

The Urban Climate AI Challenge: The 2019–2025 Test

A major Carnegie Mellon climate study (Lai & Dzombak) collected weather records for 210 U.S. cities from ~1900 up through 2018.

Your job is simple: put yourself at the end of 2018 and predict the actual yearly weather for each city from 2019 through 2025, without using any data that reveals how those years turned out. You are welcome to bring in other data too (e.g., weather records up to 2018 from other sources, census, land cover, building stock); the data rule below says what can feed your forecasts. We will compare your predictions against the weather actually recorded at airport and city weather stations during those years. Then, use what you learned to propose one policy idea that helps a city, or the country, cope with a climate risk.

Data rule (honor code): to forecast temperature, precipitation, hottest day and CDD, use only records up to 2018-12-31: the challenge dataset, or other sources cut at the same date. Do not use any 2019–2025 data that responds to the weather as an input to your forecasts: weather observations from any source, and anything driven by them (e.g., streamflow, air quality, energy demand, plant or disease data). Other data (e.g., census, land cover, energy, health) is welcome for your analysis and policy idea; list every external source in your README. Examples are in the FAQ (Data & submission tab).

Dates: November 2 – November 13, 2026.

Teams

Registration is team-based. Each participant can be in one team only.

What your model predicts

For each of the 210 cities, predict a single number for each year from 2019 through 2025 for four metrics:

Average Annual Temperature

°F

annual_avg_temp

The overall yearly average temperature.

Pittsburgh, 1900–2018

Total Annual Precipitation

inches

total_precip_in

Total liquid moisture (rain plus melted snow) recorded for the year.

Pittsburgh, 1900–2018

Hottest Day of the Year

°F

hottest_day_temp

The single highest daily peak temperature recorded that year.

Pittsburgh, 1900–2018

Cooling Degree Days (CDD)

°F-days

cdd

A single yearly number that adds up how many degrees each day's average temperature rose above 65 °F; used to estimate how much air conditioning a city needs.

Pittsburgh, 1900–2018

How CDD is computed

$$ \text{CDD} = \sum_{d=1}^{D} \max\left(0,\ \frac{T_{\max,d} + T_{\min,d}}{2} - 65\right) $$

  • D: the number of days in the year (365 or 366).
  • Tmax,d and Tmin,d: that day's highest and lowest temperatures, in °F.
  • max(0, …): a day at or below 65 °F adds nothing.

Example: a day with a high of 90 °F and a low of 70 °F averages 80 °F, so it adds 80 − 65 = 15 to the year's CDD.

No probabilities or intervals required: just your model's best guess for each value.

How the forecasts are scored

Each metric is scored with Mean Absolute Error (MAE), the average gap between your prediction and what actually happened (lower is better):

$$ \text{MAE} = \frac{1}{N} \sum_{i=1}^{N} \left| \text{Predicted}_i - \text{Actual}_i \right| $$

The four metrics use different units, so each one is compared against the 30-year-average baseline. The Score says how much better than that baseline your model was:

$$ S_m = 1 - \frac{\text{MAE}_{m,\ \text{your model}}}{\text{MAE}_{m,\ \text{30-year average}}} \qquad \text{Score} = 100 \times \max\left(0,\ \frac{S_{\text{temp}} + S_{\text{precip}} + S_{\text{hot}} + S_{\text{cdd}}}{4}\right) $$

Score = 0: no better than the 30-year average. Score = 100: a perfect forecast. Higher is better.

Live Score2019–2021
Final Score2022–2025

The two are kept separate on purpose: a model tuned to 2019–2021 does not necessarily predict later years well.

Note: a few rows are not scored: years where the station missed more than 10 days, and the few cities without enough past data for a baseline. Predict every row anyway.

Live leaderboard

Coming soon

Coming soon: the leaderboard is not open yet. It will appear in a Leaderboard tab on this Space; submissions made before then are kept and will be scored when it opens.

Your Live Score (2019–2021) can take up to 24 hours to appear on the leaderboard. Use it to check how your model is doing as you improve it; the final ranking and the winners are decided after the deadline (see Judging). If your score has not appeared after 24 hours, post in the Space's Community tab.

What to beat

Three simple reference baselines:

  1. 1Last Year Repeated

    every year from 2019 to 2025 repeats the city's last valid value (2018 for most cities).

  2. 230-Year Average

    every year equals the city's average from 1989 to 2018.

  3. 3Linear Trend

    a straight line through the last 30 years, extended out to 2025.

Target: if your model cannot beat the 30-year average, it is probably overfitting to historical noise. Hottest day and precipitation are noisy: matching the 30-year average on them is already reasonable; most of the gain is in temperature and CDD.

CDD target: beat the 30-year average (level 1), then the linear trend (level 2). In a practice run (train up to 2010, predict 2011–2018), the 30-year average scores an MAE of 213 °F-days and the trend line 177 °F-days. Differences under 10 °F-days count as a tie.

Policy idea

Write a short Policy Idea section (half a page at most) in your submission's README.

The task, in one sentence: using what you learned from the data, suggest one idea that could help a city, or the country, cope with a climate risk sustainably. Answer these three questions however you like:

  1. 1What risk did you notice?

    E.g., more cooling days, hotter peak days, wetter or drier years.

  2. 2What would you do about it, and who would do it?

    A city, a state, the federal government, a utility, a school, a business: your choice.

  3. 3Why is it sustainable?

    It helps without making emissions or costs worse in the long run.

Any city, group of cities or the whole U.S.; any risk in the data; bold ideas welcome. Inspiration: EPA Heat Island Reduction, heat.gov.

What you submit

A link to your Hugging Face dataset, containing your forecasts (submission.csv), a README with your method, policy idea and feedback to the ecosystem, and your code.

You can revise and submit as many times as you like until the deadline. Your latest valid submission counts. Details are in the Data & submission tab.

Your public Hugging Face dataset
submission.csv

Your forecasts: a number for each of the 5,880 rows

README.md

Method · Policy idea · Feedback to the Ecosystem

code/

The notebooks or scripts that produce your forecasts

Submission tab

paste the link; its current version (commit) is recorded

Score + judges

your latest submission counts

Judging

Final winners are selected after the deadline, by a panel of 3–4 judges, on six criteria:

25%
Demonstrated Reuse & Extensibility

Meaningful use or rigorous attempted reuse of ProtoStar artifacts; useful extension, diagnosis, or recommendation for future researchers. Evidence about discovery and reuse friction is scored under Feedback to the Ecosystem, not here.

20%
Interoperability & Integration

Successful combination or portability across repositories, institutions, agencies, environments, or domains, or rigorous evidence showing why integration fails; value and limits of common standards and machine-actionable formats.

20%
Reproducibility & Openness

Ability for others to reproduce the work using provided materials; for submissions declared in advance as negative results, a well-documented attempt that isolates why reproduction is not possible with the available artifact, its metadata, or its documented environment. Recommendations arising from that finding are scored under Feedback to the Ecosystem, not here.

15%
Scientific, Research, or Societal Value

Useful knowledge, new research capability, improved decision-making, or meaningful public benefit.

15%
Feedback to the Ecosystem

Specific, actionable evidence about barriers, failed reuse or reproduction, metadata issues, licensing, documentation, infrastructure, policy, or other opportunities for improvement.

5%
Technical Creativity

Novel methods, AI approaches, technical solutions, or combinations of resources that advance the goals of the challenge.

Judges consider your Score and method under Scientific, Research, or Societal Value and Technical Creativity, and your policy idea under Scientific, Research, or Societal Value.

Write your Feedback to the Ecosystem in its own section of your README: what got in the way (data, metadata, documentation, licensing, tools) and what would fix it.

If your main finding is that reuse or reproduction did not work, set Negative result: yes at the top of your README; the version recorded at your latest submission counts.

Explain not just what you built, but why it is innovative, who would use it, and whether it could actually work in the real world.

Prizes

$500 in Hugging Face credits

2

Second place team

$1,000 in Hugging Face credits

1

First place team

$250 in Hugging Face credits

3

Third place team

Top-placing CMU team

$2,500, shared among the team members

If the top-placing CMU team also places in the top three, it receives that place's Hugging Face credits as well.

1900: +1.2 °F1901: -0.1 °F1902: -0.0 °F1903: -0.8 °F1904: -0.9 °F1905: -0.7 °F1906: +0.2 °F1907: -0.5 °F1908: +0.6 °F1909: -0.2 °F1910: +0.4 °F1911: +0.8 °F1912: -1.3 °F1913: +0.5 °F1914: +0.2 °F1915: -0.1 °F1916: -0.7 °F1917: -1.9 °F1918: +0.3 °F1919: +0.3 °F1920: -0.6 °F1921: +2.4 °F1922: +0.8 °F1923: +0.2 °F1924: -1.1 °F1925: +0.9 °F1926: +0.0 °F1927: +0.7 °F1928: +0.4 °F1929: -0.4 °F1930: +0.8 °F1931: +2.6 °F1932: +0.7 °F1933: +1.7 °F1934: +2.0 °F1935: +0.3 °F1936: +0.5 °F1937: +0.2 °F1938: +2.0 °F1939: +1.8 °F1940: -0.2 °F1941: +1.4 °F1942: +0.4 °F1943: +0.1 °F1944: +0.4 °F1945: +0.1 °F1946: +1.4 °F1947: +0.2 °F1948: +0.0 °F1949: +0.9 °F1950: -0.6 °F1951: -0.6 °F1952: +0.6 °F1953: +1.6 °F1954: +1.2 °F1955: +0.2 °F1956: +0.4 °F1957: +0.4 °F1958: -0.2 °F1959: +0.4 °F1960: -0.7 °F1961: -0.2 °F1962: -0.3 °F1963: -0.1 °F1964: -0.0 °F1965: -0.2 °F1966: -0.6 °F1967: -0.4 °F1968: -0.6 °F1969: -0.6 °F1970: -0.2 °F1971: +0.0 °F1972: -0.7 °F1973: +0.8 °F1974: +0.3 °F1975: +0.1 °F1976: -0.7 °F1977: +0.4 °F1978: -1.0 °F1979: -0.9 °F1980: +0.2 °F1981: +0.7 °F1982: -0.3 °F1983: +0.0 °F1984: +0.1 °F1985: -0.5 °F1986: +1.1 °F1987: +1.5 °F1988: +0.5 °F1989: -0.3 °F1990: +1.9 °F1991: +1.6 °F1992: +0.5 °F1993: -0.3 °F1994: +0.9 °F1995: +0.6 °F1996: -0.7 °F1997: -0.2 °F1998: +2.3 °F1999: +1.5 °F2000: +0.5 °F2001: +1.2 °F2002: +1.0 °F2003: +0.5 °F2004: +0.8 °F2005: +1.4 °F2006: +2.2 °F2007: +1.6 °F2008: +0.2 °F2009: +0.2 °F2010: +1.2 °F2011: +1.5 °F2012: +3.4 °F2013: +0.5 °F2014: +0.2 °F2015: +2.4 °F2016: +3.0 °F2017: +2.6 °F2018: +1.7 °F