数据集 API

组合编码与训练数据集。流水线与数据格式见 数据流水线。

PortfolioDataEncoder

PortfolioDataEncoder(format: Literal['json', 'csv', 'binary'] = 'json', id_key: str = 'InvestorID', portfolio_key: str = 'Portfolio', proportion1_key: str = 'Proportion1', proportion2_key: str = 'Proportion2')

Encoder/decoder for portfolio data.

Reads (decodes) and writes (encodes) portfolio data in JSON, CSV, or binary format. Each record holds an ID, a string list of asset names/codes, and the corresponding holding-proportion lists.

Parameters:

Name	Type	Description	Default
format	Literal['json', 'csv', 'binary']	Data format; one of "json", "csv", "binary".	'json'
id_key	str	Name of the ID field.	'InvestorID'
portfolio_key	str	Name of the holding asset name/code field.	'Portfolio'
proportion1_key	str	Name of holding-proportion field 1.	'Proportion1'
proportion2_key	str	Name of holding-proportion field 2.	'Proportion2'

encode

encode(df: DataFrame, file_path: str | Path, include_proportion: bool = False) -> None

Encode a DataFrame and save it to a file.

decode

decode(file_path: str | Path, include_proportion: bool = False) -> DataFrame

Decode from a file and return a DataFrame.

Word2VecDataset

Word2VecDataset(path: Union[str, Path, List[str]], format: str = 'csv', id_key: str = 'InvestorID', portfolio_key: str = 'Portfolio')

Word2Vec training dataset.

Initialize the Word2Vec dataset.

Parameters:

Name	Type	Description	Default
path	Union[str, Path, List[str]]	Dataset path — a file, a folder, or a list of file paths.	required
format	str	Data format; "csv" or "json".	'csv'
id_key	str	Name of the investor-ID field.	'InvestorID'
portfolio_key	str	Name of the holdings field.	'Portfolio'

AssetBERTMLMDataset

AssetBERTMLMDataset(path: Union[str, Path, List[str]], tokenizer: PreTrainedTokenizerFast, max_length: int = 20, mask_prob: Optional[float] = 0.15, mask_indices: Optional[List[int]] = None, format: Literal['csv', 'json'] = 'csv', include_proportion: bool = False, cache_size: int = 10, num_repeats: int = 1, id_key: str = 'InvestorID', portfolio_key: str = 'Portfolio', proportion1_key: str = 'Proportion1', proportion2_key: str = 'Proportion2')

Bases: Dataset

Asset BERT MLM training dataset (multiprocessing-capable).

Initialize the Asset BERT MLM dataset.

Parameters:

Name	Type	Description	Default
path	Union[str, Path, List[str]]	Data file, folder path, or list of files.	required
tokenizer	PreTrainedTokenizerFast	Pretrained tokenizer.	required
max_length	int	Maximum sequence length.	20
mask_prob	Optional[float]	MLM masking probability (for BERT training).	0.15
mask_indices	Optional[List[int]]	Fixed mask positions (for ASMP training and testing).	None
format	Literal['csv', 'json']	Data format.	'csv'
include_proportion	bool	Whether to include holding proportions.	False
cache_size	int	File-cache size.	10
num_repeats	int	Dataset repeat factor (extends the training-set length to reduce frequent epoch restarts on small samples).	1
id_key	str	Name of the investor-ID field.	'InvestorID'
portfolio_key	str	Name of the holdings field.	'Portfolio'
proportion1_key	str	Name of holding-proportion field 1.	'Proportion1'
proportion2_key	str	Name of holding-proportion field 2.	'Proportion2'