The electricity system is the lifeline of any modern economy. After roughly 120 years of existence, it is in the early stages of receiving a significant makeover, posing enormous challenges and opportunities. Three unmistakable trends, driven by economics as well as policies on greenhouse gas emissions, are causing this paradigm shift – deep penetration of natural gas; development and integration of large, utility-scale renewables; and increasingly widespread use of distributed energy devices. Energy efficiency and conservation have had a long history of significant impact and are exceedingly important for deep decarbonization; however, we will leave that topic for a later article. Here we focus on the grid.
Impact of natural gas
Natural gas is inexpensive and abundant in the U.S., and the turbines that convert its thermal energy into power are more than 60 percent efficient, making electricity from natural gas very cheap. This is displacing electricity from coal-fired power plants, which is reducing particulate air pollution as well as carbon dioxide and other greenhouse gas emissions. Unfortunately, natural gas is also economically displacing carbon-free electricity from nuclear energy. When combined with the aging of our existing nuclear plants, and the lack of sufficient new build out, we face the regrettable future of declining electricity from nuclear power. If we are to deeply decarbonize our grid in the long term, it would be prudent for our nation to keep a diversity of options, including nuclear. The option for economically competitive nuclear power requires a well thought-out roadmap, which is now being formulated.
High penetration of utility-scale renewables
The second trend towards deep decarbonization is integrating on the grid electricity produced by large-scale renewable sources. Wind is the most inexpensive way to produce electricity today, whereas that from solar is steadily decreasing to the point that it will likely be at parity without subsidies with natural gas by end of this decade. California has passed legislation requiring that by 2030 our grid will integrate 50 percent of its electrical energy from renewable sources. Many other regions in the world are also on a similar roadmap. Most of this will be achieved via large-scale wind and solar farms that deliver electricity directly to the transmission network . Integrating solar and wind power is challenging because they often ramp up or down several gigawatts of electrical power within an hour. Our current Tesla-Edison grid was never designed for such transient sources of electricity.
Balancing this transient electrical power, which in much of California and the U.S. is the job of independent system operators (ISOs) and regional transmission organizations (RTOs) that function across multiple states, is rather challenging. What are the options? When renewable electricity generation is so cheap, what matters is the cost of integration. Perhaps the most inexpensive is to build long-distance, ultrahigh-voltage transmission lines  , which takes advantage of the fact that electricity can be shared across those regions sufficiently far away that the transients are uncorrelated. Significant strides have been made in the U.S. in building new transmission. However, permitting and siting these transmission lines is a long, drawn-out and expensive process, requiring resolutions between multiple federal and state agencies, and extensive analysis of both need and cost-allocation. Strengthening and streamlining this process is urgently needed, which requires resources to support analysis for decision makers.
The second option is to use electricity storage or natural gas plants for balancing. Today, the predominant and the most inexpensive way to store electricity is via pumping water up a hydroelectric dam, but this remains geographically limited with sometimes socially unacceptable environmental impact. While battery costs are steadily reducing, the key question for both batteries and natural gas plants is: What is the minimum capacity required to balance the transients? The minimum capacity translates to minimizing costs. The third option is to have the loads dynamically track the transients of the generation. This is exactly opposite of what we have used for the last 120 years of the grid, where the generation always tracks the load. Most likely, a combination of all three options would be required. The key will be to find the combination that minimizes costs for society, while achieving the reliability and environmental benefits we expect from the new power system.
Regardless of the approach, the presence of transients requires quick and judicious action for balancing at the least possible cost of integration. The overlay and ubiquitous use of an enabling information infrastructure consisting of power electronics, communication, computing and control systems will be one of the salient features of modernizing the Tesla-Edison grid. But it would also require new wholesale market structures and pricing mechanisms for these balancing assets to be utilized and financially compensated. The presence of abundant zero marginal cost energy (wind and solar) and storage violates key assumptions that underlie the current wholesale market structure. The introduction of new markets to aggregate energy resources on the demand side is already underway, and we are likely to see even more in the future. The Supreme Court’s recent decision also provides much needed support for such demand-response programs.
Distributed energy resources
If the previous issues sound challenging, here is an even thornier problem. With decreasing costs, innovative financing and favorable customer experience, consumers are finding it increasingly attractive from an economic and environmental viewpoint to install solar photovoltaic (PV) panels on their rooftops and own or lease electric vehicles (EVs). Consumers are also buying and integrating smart thermostats that control heating and cooling in their homes. The use of these new energy services is driven by consumer choice, and sometimes made lucrative with government incentives. The key point to note is that these devices reside behind the meter. They are also networked to their respective private cloud datacenters (e.g., Nest and Honeywell for smart thermostats, Nissan and Tesla for EVs, etc.) where consumer preferences, schedules and activities are closely monitored and recorded. Cloud connectivity allows the devices to be remotely monitored and controlled in an automated way. The private cloud can also aggregate the load or generation of an ensemble of these devices behind the meter to sell valuable balancing services on the wholesale market in exchange for a revenue stream.
Here is the challenge. Despite the fact that the EVs, PVs and thermostats behind the meter generate or consume large amounts of power, the utility is largely blind to their presence and/or activity, because a utility’s domain stops at the meter. Yet, since a home is tied to a utility electric line, turning on and off these devices behind the meter can either push or draw large amounts of power on the utility line. If nothing is done, this push and pull of power can lead to reliability problems for multiple homes on the utility line. There are engineering solutions to such problems, but the utilities are not often incentivized to use the technology. And private companies selling the devices and services behind the meter are not incentivized to do anything on the utility line. It is a power struggle, both literally and figuratively.
Unfortunately, the only mechanism of engagement across the meter available today is “net energy metering (NEM)” used for electricity generation behind the meter, which is a rather blunt instrument. Some versions of NEM require the utility to buy the electricity generated by the home at the utility’s retail price, which is attractive to the consumer and companies behind the meter but heavily contested by the utilities, since they could have purchased the same at a cheaper rate from the wholesale market through bilateral contracts or via self-generation. And, consumer advocates see such pricing of NEM as providing significant subsidies that are paid for by “non-participating” consumers, often low-income. Many utilities, not all, are trying to block and tackle using a variety of means, including political ones, to increase the costs for rooftop PV or setting a limit on penetration within a service area. If this is an issue now, the situation will get aggravated further in the future with the increasing use of energy devices and services behind the meter. Unfortunately, innovation in the consumer energy space will likely get caught in the crossfire. If this seems confusing and convoluted, let us reassure the reader – it is indeed so. We desperately need a new policy framework that is simple, transparent and fair to both sides of the meter.
What is absent from the traditional utility regulatory framework is that there are multiple opportunities for avoided costs to the utilities if the devices behind the meter can provide services that the utility needs. For example, a solar or a battery inverter can easily be used for Volt-VAR control on a distribution network, which the utility values. This is not practiced today because there is no mechanism for the utility and the private companies that own/operate the inverter to engage in a meaningful way. Looking ahead, one can envision a scenario where there could be a variety of devices behind the meters of multiple homes connected by utility lines. If, for instance, a PV on one rooftop has excess power, one could slightly increase the charging rates across multiple EVs or water heater on the same utility line. If the consumption or generation schedules for devices behind the meter can be shared by the private clouds with the utility, the utility could plan and operate its assets much more judiciously, thus reducing the cost to the consumers, who eventually have to pay for the utility investments through increases in rates. Current retail rate structures (either tiered or time-of-use) and NEM that are regulated by the public utility commissions are inadequate because they are not designed to recognize this value.
What we need is a transparent mechanism of engagement across the meter, between utilities on one side and the private companies (EVs, PVs, thermostats and others) on the other. One approach may be the formation a distribution system operator (DSO), which is being considered by some states. We ought to explore other options as well. Regardless of what we call it, the engagement mechanism must look at the technology trends for both energy and computing. It should identify services provided across the meter, respecting the grid constraints and valuing costs, both real and avoided. It would allow valuation and pricing of these services, and financial transactions between the entities based on the services. It would also involve the protocols for data sharing that respect the proprietary nature of private enterprises and privacy of consumers, but allow sufficient information to be shared so that energy services can be offered to the consumer and between entities across the meter. Finally, it would also define how these private cloud entities and utilities would engage with the ISOs/RTOs and other grid operators, so that the act of providing ancillary services in the wholesale market does not destabilize the distribution grid or ignore consumer preferences.
It is worth noting that the engagement mechanisms at the wholesale markets operated by ISOs/RTOs are transparent and well understood. They involve multiple players involving loads, generation or both. The pricing mechanisms consider not only the fuel costs, but also congestion on the transmission lines and contingency plans. There is no reason why the engagement at the distribution level cannot be framed in a similar way for a variety of services across the meter and to the consumer. While the framework for engagement is roughly outlined above, the actual mechanism could be enacted via multiple viable options - e.g., market-based pricing versus fixed pricing for various services; application program interfaces (APIs) for data sharing for a variety of services. Each such option will require deep and objective analysis to understand the ramifications and pros and cons, as well as costs and benefits.
Given the importance of this issue for decarbonizing our grid while offering new clean energy services for the consumer, now is the time for the ecosystem to come together and identify and analyze these options.
Acknowledgments: The authors would like to thank Dian Grueneich, Stephen Comello and Michael Wara of Stanford University for their comments and suggestions.
 About 5-6 cents per kilowatt-hour.
 From about 50 percent a decade ago to about 25-30 percent today.
 Follow the work of the Secretary of Energy Advisory Board Task Force on Future of Nuclear Power.
 While electricity from hydroelectric plants is inexpensive (3-4 cents per kilowatt-hour) and has been around for decades, it is limited geographically. Nevertheless, new hydroelectric plants are being explored, permitted and built.
 Power purchase agreements have been signed in the range of 2-3 cents per kilowatt-hour, which includes a federal production tax credit of 2.3 cents per kilowatt-hour
 Power purchase agreements have been signed as low as 4 cents per kilowatt-hour, which includes a federal investment tax credit of 30 percent.
 California Senate Bill 350, October 2015. California, like many governments, includes only renewables on the utility-side of the meter in its definition of a Renewable Portfolio Standards (RPS). Thus, we address rooftop solar on the customer-side of the meter in our discussion below of distributed energy resources.
 Biomass, existing hydroelectric and geomthermal will also play important roles.
 This is equivalent to turning on or off a few nuclear plants within an hour.
 This is the approach taken by China, which leads the world in ultrahigh voltage technology.
 For example in 2013, investor-owned utilities in the U.S. invested a record $17 billion in new transmission. These utilities forecast up to another $48 billion in transmission investments over the next 10 years, with projects supporting the integration of renewable resources representing about one-half, at $22.1 billion.
 Natural gas turbines can ramp up or down at about 50 megawatts per minute, but involve carbon dioxide emissions.