Utilization of Existing ENTSO-E Transparency Platform Data
This action can be carried out by third parties who are actively playing a role in the development of flexibility markets or start-ups that are aiming to become a market player. Third parties from the fields of energy service provision and other platforms aiming to be part of TSO-DSO-Consumer value chain are welcome to apply.
The participants in the Open Call will have the opportunity to get in-depth knowledge of the mechanisms used in the IEGSA platform. That will provide an excellent base to evaluate valuable market information that could be made transparent at the moment or potentially other information that could go into proposals for regulatory amendments in order to extend the mandatory reported information in the future. The participants can benefit from the opportunity to develop business models and pilot platforms that can exploit the transparency information for flexibility markets.
As flexibility markets improve and become an important part of the energy markets, more information is expected to become available and its accessibility should be ensured to all market players in a transparent manner. In order to achieve that, there is a need to specify these requirements from the perspective of all market players in addition to NRAs. As an added benefit, the enhancement of market transparency will help other external market players to better position themselves in the market. Therefore, the focus of this Business Use Case is to provide a comprehensive frame for available dataset, indicating the regions that are in need of flexibility products and hence, this will create new business opportunities for flexibility system providers.
Specification of the requirements for transparency of flexibility markets shall take into account all the participants within the market. This should include system operators, flexibility service providers, consumer/prosumer groups, market operators, energy platforms and etc. Workshops and surveys may be facilitated among such players, if needed. By having a more transparent flexibility market, the following benefits can be achieved:
- More flexibility service providers may be attracted and trigger more investments. This will result in a more efficient and well-functioning market;
- Market players will better value their services;
- Attracting more players will increase reliability and security of supply;
- Providing investment signals where there is a need for increased flexibility capacity;
- Efficient congestion management can defer the need for grid enhancements;
- Having more efficient, liquid and competitive flexibility markets.
In order to achieve the benefits listed above the participants in the Open Call are requested to investigate business case for Utilization of existing data on the Transparency Platform. The main purpose is to develop solutions that utilize the existing data on the platform in a way which results in usable, well visualized information for flexibility service providers. In particular, the project should address one or more of the following challenges:
- Creating a common ground for all participants and benefit from utilizing revealed information from the perspective of flexibility markets and congestion management;
- Providing useful information on grid level, without compromising sensitive data from the perspective of system operators;
- Defining a common dataset that will serve to improve transparency in an environment with diversified flexibility products;
- Encouraging prototypes/pilots that gather and publish transparency data for flexibility markets or flexibility for congestion management;
- Provide recommendations for changes from regulatory point of view.
Available Data on Transparency Platform
In accordance with Regulation 543/2013, the ENTSO-E Transparency Platform was launched on 5 January 2015. Following the launch of the new platform, the www.entsoe.net website, on which TSOs had voluntarily published some market data since 2011, was de-commissioned in March 2015. The historical data from 2011-2014, which was previously published on entsoe.net, is available to download from the Transparency Platform > Data Pre-5.1.15 section at the following link:
Currently, data on Transparency Platform is published under 7 different domains.
Load: Within this domain, actual total load for and load forecast data with various horizon (day, week, month, year) is presented.
Generation: Data regarding installed capacities, generation and generation forecast is presented.
Transmission: Data about cross border power transfers and forecasted capacities is revealed.
Balancing: Under this domain, data regarding to keep the electricity grid balanced is published. This includes bids data, accepted offers and activated reserve information including prices besides balancing state of the areas.
Outages: Within this domain, data regarding planned maintenances and forced outages in the grid is published.
Congestion Management: Data about actions taken to relieve overloaded parts of the transmission grid is published.
System Operations: Data about operational agreements and frequency quality is published.
Data Download Options from Transparency Platform
TP offers various ways of data export alternatives. In order to meet the different user needs, followings options are available:
- Web GUI – Transparency Platform (https://transparency.entsoe.eu/ )
- SFTP – User Guide (https://transparency.entsoe.eu/content/static_content/Static content/knowledge base/SFTP-Transparency_Docs.html )
- Restful API – User Guide
(https://transparency.entsoe.eu/content/static_content/Static content/web api/Guide.html )
- Data Repository Solution – User Guide (https://transparency.entsoe.eu/content/static_content/Static content/data repository/DataRepositoryGuide.html )
- Subscription Service – User Guide (https://transparency.entsoe.eu/content/static_content/download?path=/Static%20content/knowledge%20base/Subscription%20Configuration%20User%20Guide%20v0.1.pdf )
Users who are interested in limited amount of data can directly use GUI export option. On the other hand, SFTP is suitable for bulk data downloads but data available on SFTP refresh once every hour. On the contrary, Restful API can serve for the ones who are interested in the most recent updates on data. But there are also some limitations apply to API requests in terms of the number of requests per minute, the number of files to be downloaded per request and the time window allowed for queries depending on the data item of interest. The last option, Data Repository Solution allows download up to 50 MB. Requests are processed in the background, asynchronously, without imposing a load on the platform through a preferred channel for communication (Web service or ECP). Finally, the platform allows users to subscribe for a data feed in which the platform pushes updates to the user’s endpoint through a web service or ECP channel.
|Download Option||File Type||Data Updates|
|Web GUI||xml, csv, xlsx||Almost real time|
|Restful API||xml||Almost real time|
|Data Repository||xml (zipped)||Almost real time|
|Subscriptions||xml||Almost real time|
Conditions for Use of Transparency Platform Data
Conditions for use of TP data is defined by the Terms and Conditions within a dedicated section as follows:
“In accordance with the applicable legislation, the Data User shall, when using of the Transparency Platform Data for any purpose whatsoever:
- Use the Transparency Platform Data in good faith and always comply with good business practices regarding the re-use of publicly available data;
- Mention the ENTSO-E Transparency Platform as the source of publication of the data, in accordance with good industry practices and comply with all reasonable requests from ENTSO-E regarding the visibility of the ENTSO-E Transparency Platform origin of the re-used Transparency Platform Data;
- Be only allowed to make reference to the ENTSO-E Transparency Platform as the source of publication of the re-used data. It is therefore expressly prohibited to use the ENTSO-E Transparency Platform name or the ENTSO-E name in any manner that is likely to cause confusion regarding the possible existence of any kind of sponsorship or of endorsement of any use of the Transparency Platform Data by the Data User;
- Not cause prejudice to the copyright or related right on a Transparency Platform Data, which may be owned by the concerned Primary Owner of Data. In case of a risk to cause prejudice to said right, the Data User shall seek the prior agreement of the holder of the copyright or related right. Notwithstanding this requirement, as a facilitation for the Data User, ENTSO-E publishes on the Transparency Platform and regularly updates the list of the Transparency Platform Data which can be freely re-used with no need to seek for the prior agreement of the respective Primary Owner of Data. The Data User has responsibility to check this list before each re-use of the Transparency Platform Data.”
The complete TP Terms and Policy documentation is available on the following link: https://transparency.entsoe.eu/content/static_content/download?path=/Static%20content/terms%20and%20conditions/181108_ENTSOE_Transparency_Terms_Conditions_MC_APPROVED.pdf
The participants in the Open Call are requested to investigate business case for Utilization of existing data on the Transparency Platform and propose to develop solutions that utilize the existing data on the platform in a way which results in usable, well visualized information for flexibility service providers. Revealed data should include at least the below information:
- Flexibility needs of areas for balancing services
- Identification of possible congestion in transmission lines
- Identification of areas that may require demand response services
- Price distribution of the procured balancing reserves
The proposed business use case address transparency of flexibility markets for the INTERRFACE on general level.
Important information for applicants
Conditions for Use of Transparency Platform Data
The information published by ENTSO-E is submitted by data providers such as TSOs, power exchanges, allocation offices or other qualified third parties. Detailed information regarding the data published under above mentioned domains is available in the “Detailed Data Descriptions” document as referenced in the Manual of Procedures that can be found here: https://www.entsoe.eu/data/transparency-platform/mop/
Manual of Procedures (MoP), as required in Article 5 of Regulation 543/2013 is a technical guide, developed by ENTSO-E following the discussions with Stakeholders, a Public Consultation and review by ACER. The high-level description of MoP is given as follows:
- Details and format of the submission of data by primary owners to TSOs;
- Standardized ways and formats of data communication and exchange between primary owners of data, TSOs, data providers and ENTSO-E;
- Technical and operational criteria which data providers would need to fulfil when providing data to the central information transparency platform;
- Appropriate classification of production types referred to Articles 14(1), 15(1) and 16(1) of the Regulation.
MoP particularly addresses primary owners of data, data providers and data consumers.
Pursuant to Article 5 of the General Terms and Conditions of Use of ENTSO-E Transparency Platform, ENTSOE the list of data which can be freely re-used by the Data Users with no need to seek for the prior agreement of the respective Primary Owner of Data (Transmission System Operators and Transmission Capacity Allocators) can be accessed via this link:
Added value on INTERRFACE project
The liberalization process of the electricity market in European Union brought up, among other things, new requirements related to the market transparency as the competitive pan-European electricity market cannot work efficiently without its participants having access to all information relevant for their decision-making.
Transparency is essential for the creation of efficient, liquid and competitive energy markets. It is also critical for creating a level playing field between market participants and avoiding the scope for market power (if it exists) to be abused. Market transparency enables the provision of the required electricity market information for the future and further facilitates the development of efficient and competitive energy markets across Europe.
One of the most crucial part of the market transparency requirement is the data publication. This requirement is addressed by the EU regulation, (EU) No 5 43/2013 of 14 June 2013 on submission and publication of data in electricity markets. Through this regulation, it has become mandatory for European Member State data providers and owners to submit fundamental information through the ENTSO-E Transparency Platform (TP).
Incorporation of Third Parties in particular for end-users flexibility
This Business Use Case is foreseen to take advantage of existing data sets in TP by several parties such as system operators, flexibility service providers, consumer/prosumer groups, market operators, energy platforms. This will enhance transparency for flexibility markets and more service providers may be attracted and trigger more investments, enabling also the way (e.g. more aggregators more end-users participation) for more efficient and well-functioning market.