Price Data Compilers

NEMStorageUnderUncertainty.ActualData — Type

struct ActualData{T<:AbstractFloat} <: NEMStorageUnderUncertainty.NEMData

region::String
times::Vector{Dates.DateTime}
prices::Vector{T} where T<:AbstractFloat
τ::AbstractFloat

A data structure used to store actual price data and metadata

source

NEMStorageUnderUncertainty.ForecastData — Type

struct ForecastData{T<:AbstractFloat} <: NEMStorageUnderUncertainty.NEMData

region::String
run_times::Vector{Dates.DateTime}
forecasted_times::Vector{Dates.DateTime}
prices::Vector{T} where T<:AbstractFloat
τ::AbstractFloat
run_time_aligned::Bool

A data structure used to store forecast price data and metadata

run_time_aligned indicates whether PD and P5MIN raw data used to construct a ForecastData instance were aligned along actual_run_time, i.e. same start and end actual_run_time.

source

NEMStorageUnderUncertainty.get_ActualData — Function

get_ActualData(
    actual_data::DataFrames.DataFrame,
    region::String
) -> NEMStorageUnderUncertainty.ActualData
get_ActualData(
    actual_data::DataFrames.DataFrame,
    region::String,
    actual_time_window::Union{Nothing, Tuple{Dates.DateTime, Dates.DateTime}}
) -> NEMStorageUnderUncertainty.ActualData

Summary

Get an ActualData instance.

Arguments:

actual_data: DataFrame generated by get_all_actual_data
region: Market region in the NEM
actual_time_window: Tuple used to filter DataFrame

Returns:

An ActualData instance.

Methods

get_ActualData(actual_data, region)
get_ActualData(actual_data, region, actual_time_window)

defined at /home/runner/work/NEMStorageUnderUncertainty/NEMStorageUnderUncertainty/src/data.jl:83.

source

NEMStorageUnderUncertainty.get_ForecastData — Function

get_ForecastData(
    pd_data::DataFrames.DataFrame,
    p5_data::DataFrames.DataFrame,
    region::String
) -> NEMStorageUnderUncertainty.ForecastData
get_ForecastData(
    pd_data::DataFrames.DataFrame,
    p5_data::DataFrames.DataFrame,
    region::String,
    run_time_window::Union{Nothing, Tuple{Dates.DateTime, Dates.DateTime}}
) -> NEMStorageUnderUncertainty.ForecastData
get_ForecastData(
    pd_data::DataFrames.DataFrame,
    p5_data::DataFrames.DataFrame,
    region::String,
    run_time_window::Union{Nothing, Tuple{Dates.DateTime, Dates.DateTime}},
    forecasted_time_window::Union{Nothing, Tuple{Dates.DateTime, Dates.DateTime}}
) -> NEMStorageUnderUncertainty.ForecastData

Summary

Get a ForecastData instance.

Imputes PREDISPATCH data. Specifically, this involves:

Backwards filling forecasted_time
- A forecast at the end of a settlement period should reflect previous 5 intervals
- e.g. price at 13:30 is back filled for 13:25, 13:20 ... 13:05
Forward filling run_time
- Latest forecast should be used until a new forecast is run
- e.g. prices from run at 13:30 are used for 13:35, 13:40 ... 13:55
- We then need to remove periods when run_time < forecasted_time

Imputation is carried out using Impute.jl.

Arguments:

pd_data, p5_data: DataFrames generated by get_all_pd_and_p5_data
region: Market region in the NEM
run_time_window: Tuple used to filter DataFrame based on run times
forecasted_time_window: Tuple used to filter DataFrame based for forecasted time

Returns:

An ForecastData instance.

Methods

get_ForecastData(pd_data, p5_data, region)
get_ForecastData(pd_data, p5_data, region, run_time_window)
get_ForecastData(
    pd_data,
    p5_data,
    region,
    run_time_window,
    forecasted_time_window
)

defined at /home/runner/work/NEMStorageUnderUncertainty/NEMStorageUnderUncertainty/src/data.jl:317.

source

NEMStorageUnderUncertainty.get_all_actual_data — Method

get_all_actual_data(path::String) -> DataFrames.DataFrame

Summary

Obtains actual data from parquet files located at path

Arguments

path: Path to parquet partitions

Returns

DataFrame with settlement date, region and corresponding energy prices

Methods

get_all_actual_data(path)

defined at /home/runner/work/NEMStorageUnderUncertainty/NEMStorageUnderUncertainty/src/data.jl:51.

source

NEMStorageUnderUncertainty.get_all_pd_and_p5_data — Method

get_all_pd_and_p5_data(
    pd_path::String,
    p5_path::String
) -> Tuple{DataFrames.DataFrame, DataFrames.DataFrame}

Summary

Obtains and compiles all forecasted price data from parquet files located at the P5MIN path (p5_path) and the PREDISPATCH path (pd_path)

Note that Parquet.jl cannot parse Timestamps from .parquet, so we use unix2datetime.

Arguments

pd_path: Path to PREDISPATCH parquet partitions
p5_path: Path to P5MIN parquet partitions

Returns

Compiled forecast data with nominal run times, forecasted times, regions and their corresponding energy prices.

Methods

get_all_pd_and_p5_data(pd_path, p5_path)

defined at /home/runner/work/NEMStorageUnderUncertainty/NEMStorageUnderUncertainty/src/data.jl:130.

source