Test Prep Professionals

(US News) By Robert Morse and Matt Mason–Now in its third year, the US News/Ratheyon STEM Index tracks key economic- and education-related science, technology, engineering and mathematics activity in the United States over time.

Just like other well-known indexes — the S&P 500 Stock Index, for example, or the Consumer Price Index — the U.S. News/Raytheon STEM Index has a base year from which changes in the underlying factors are measured. Our index’s base year is 2000 and was set to equal 100.0. The U.S. News/Raytheon STEM Index is calculated annually by U.S. News and measures the yearly changes in the activity of its components in relation to the year 2000.

In 2016 the U.S. News/Raytheon STEM index was revised in order to better measure STEM activity from 2000 to 2015, to take into account new STEM ranking factors and to reflect changes to the existing data. Created with support from Raytheon, the Index is not a comprehensive measure of all STEM economic or STEM education activity in the United States but a snapshot in time of measurements of key indicators of STEM activity.

In order to be included in the index, a specific STEM measure generally had to be published annually starting in 2000 and had to have enough statistical significance to credibly reflect an important trend in STEM. The first U.S. News/Raytheon STEM Index relied on data from the U.S. Bureau of Labor Statistics, the National Center for Education Statistics, the College Board, the National Research Center for Colleges & University Admissions, the ACT, the International Association for the Evaluation of Educational Achievement and the Organization for Economic Co-operation and Development. Just like other widely followed indexes, U.S. News will likely make changes to the weights and components for future indices as more numerous and refined STEM indicators become available.

How the STEM Index Was Calculated

U.S. News gathered thousands of data points to create 18 subindices. Each subindex was weighed to compute the nine STEM component areas, which are the variables used to calculate the overall STEM Index. Those nine component areas measure changes in STEM employment in the United States, STEM degrees granted at the college undergraduate and graduate levels, interest in STEM at the high school level, performance on the ACT, SAT, Advanced Placement, National Assessment of Educational Progress tests and how U.S. students perform internationally using the Program for International Student Assessments and Trends in International Math and Science Studies.

The final step in the computation of the overall STEM Index was assigning separate percentage weights to each of the nine STEM component indices. The overall STEM Index is the sum of the weighted values of the nine component indices. The highest weights were given to the broadest measures of STEM: STEM employment and STEM degrees granted at the college and graduate level. Those two component indicators, which are reflective of STEM in the economy and the education process, account for 55 percent of the overall STEM Index. The other factors, which relate to education at the K-12 level, account for the remaining 45 percent of the overall STEM Index.

Employment Index (35 percent of the overall STEM Index): The Employment Index is an outcome indicator for STEM employment in the U.S. economy. It measures the proportion of STEM jobs in the U.S. by taking the number of STEM jobs divided by the total number of all jobs. We then categorized the STEM jobs into two weighted indicators: STEM employment (60 percent of the component’s weight) and STEM-related employment (40 percent). STEM employment includes science, technology, engineering and mathematics. STEM-related employment includes health, architecture and some sales occupations. We based our definition of a STEM occupation off the Office of Management and Budget’s recommend list. The 2015 data were collected from the Bureau of Labor Statistics’ Occupational Employment Statistics program.

Degrees Granted Index (20 percent): This index measures the trend in the proportion of STEM degrees as a share of total degrees by taking the number of STEM degrees granted divided by the number of all degrees granted. This breaks down into STEM associate degrees (20 percent of score), STEM bachelor’s degrees (40 percent) and STEM graduate degrees (40 percent). We relied on the Department of Homeland Security’s STEM-Designated Degree Program List to determine which degrees were in STEM fields. Due to 2015 data not being available as of April 2016, the Degrees Granted Index uses an imputed value for 2015; it is based on the percentage change between 2013 and 2014 for the 2015 STEM index score. The 2015 data were gathered from the National Center for Education Statistics.

Advanced Placement (AP) Index (13 percent): Of the 34 AP courses available, 10 of them are defined by the College Board as STEM tests. The AP Index measure has two subindices: the proportion of total AP tests that were taken in STEM fields (25 percent of score) and the proportion of total STEM AP tests taken that had a score of 3 or higher (75 percent). A score of 3 or higher generally qualifies as college credit at many universities and therefore merits more weight in our index. The AP Physics B Exam was discontinued in 2014 and replaced with AP Physics 1 and AP Physics 2 exams in 2015, a change reflected in our Index. The 2015 data were collected from College Board.

High School Interest Index (11 percent): To calculate this index, we used data from My College Options, the nation’s largest college planning program operated by the National Research Center for Colleges & University Admissions, which surveys high school students to obtain their attitudes toward STEM.

SAT Index (6 percent): Looking at college admissions test scores gives us an opportunity to track how prospective students scored in mathematics over time. The SAT index measures the average SAT math scores among all students. The 2015 data were collected from the College Board.

ACT Index (6 percent): The other major college admission test used in the United States is the ACT. The ACT Index measures the average ACT math and science scores among all students. The 2015 data were provided to us by ACT.

National Assessment of Educational Progress (NAEP) Index (3 percent): The NAEP is an assessment test that measures representative samples of 4th and 8th graders’ proficiency in a variety of subjects. We focused on the average NAEP math scores, which have been collected by the National Center of Educational Statistics biannually since 2001. We broke the data down by national average (40 percent of score) with both gender (40 percent) and ethnic (20 percent) data also receiving substantial weight.

Program for International Student Assessments (PISA) Index (3 percent): The PISA is an assessment test administered by the Organization for Economic Co-operation and Development; it measures the abilities of 15-year-old students in math, science and reading. There are 34 countries participating in the assessment, which is administered every three years. The PISA Index is based on data from the OECD and is calculated by taking the average US score in math and science and dividing it by the average OECD score. Due to revisions in the PISA framework, math data have been measured as of 2003 and science data as of 2006. The 2015 PISA data are carried over from data gathered in 2014.

Trends in International Mathematics and Science Study (TIMSS) Index (3 percent): The TIMSS assessment, which was added to the U.S. News/Raytheon STEM Index in 2015, is an international examination administered every four years, covering math and science for 4th and 8th grade students. Like PISA, the TIMSS Index is calculated by taking the average U.S. score and dividing it by the average TIMSS score. Data for the TIMSS Index was available for 2003, 2007, 2011 and 2015.