SEGB: Self-Evolving Generative Bidding with Local Autoregressive Diffusion

📝 Introduction

We propose a flexible and practical Generative Auto-bidding scheme using post-training Search, termed SEGB, to refine a base policy model’s output and adapt to various advertisers’ preferences. Our online A/B test on the Jingdong advertising platform demonstrate the effectiveness of SEGB, achieving significant improvements, e.g., 11.60% increment of target cost.🎉🎉🎉

💾 Installation

Python Environment

conda create -n segb_env python=3.9.12 pip=23.0.1
conda activate segb_env
pip install -r requirements.txt 

Prepare the Datasets

1) AuctionNet Dataset

https://alimama-bidding-competition.oss-cn-beijing.aliyuncs.com/share/autoBidding_aigb_track_data_period_7-8.zip
https://alimama-bidding-competition.oss-cn-beijing.aliyuncs.com/share/autoBidding_aigb_track_data_period_9-10.zip
https://alimama-bidding-competition.oss-cn-beijing.aliyuncs.com/share/autoBidding_aigb_track_data_period_11-12.zip
https://alimama-bidding-competition.oss-cn-beijing.aliyuncs.com/share/autoBidding_aigb_track_data_period_13.zip
https://alimama-bidding-competition.oss-cn-beijing.aliyuncs.com/share/autoBidding_aigb_track_data_trajectory_data.zip
https://alimama-bidding-competition.oss-cn-beijing.aliyuncs.com/share/autoBidding_aigb_track_data_trajectory_data_extended_1.zip
https://alimama-bidding-competition.oss-cn-beijing.aliyuncs.com/share/autoBidding_aigb_track_data_trajectory_data_extended_2.zip

2) AuctionNet-sparse Dataset

https://alimama-bidding-competition.oss-cn-beijing.aliyuncs.com/share/final/autoBidding_aigb_track_final_data_period_7-8.zip
https://alimama-bidding-competition.oss-cn-beijing.aliyuncs.com/share/final/autoBidding_aigb_track_final_data_period_9-10.zip
https://alimama-bidding-competition.oss-cn-beijing.aliyuncs.com/share/final/autoBidding_aigb_track_final_data_period_11-12.zip
https://alimama-bidding-competition.oss-cn-beijing.aliyuncs.com/share/final/autoBidding_aigb_track_final_data_period_13.zip
https://alimama-bidding-competition.oss-cn-beijing.aliyuncs.com/share/final/autoBidding_aigb_track_final_data_trajectory_data_1.zip
https://alimama-bidding-competition.oss-cn-beijing.aliyuncs.com/share/final/autoBidding_aigb_track_final_data_trajectory_data_2.zip
https://alimama-bidding-competition.oss-cn-beijing.aliyuncs.com/share/final/autoBidding_aigb_track_final_data_trajectory_data_3.zip

After download, you should concat them to a full dataset local file.

🚀 Get Started

Step 1 Train trajectory planning model

We implement a local diffusion model as our trajectory planning strategy. You could simply run by

python run/train_local_diffusion.py

Step 2 Train multiple QTs (Transformer-based Q-value critics)

We could train multiple Q-value critics by randomly run multiple times and save different models. You could run simply run by

python run/train_dt_critics.py --baseline_method 'dt_reweight_search_Q' --reweight_w 0.2 --data_path path/to/local/dataset

where reweight could simulate different preference (value_reward + reweight_w * cpa_reward, a higher reweight_w means more preference on the cpa.)

Step 3 Train action decision model w/o grpo by your choice

We choose the decision tranformer as our action decision model. You could simply run by

python run/train_dt_grpo_out_version.py

Step 4 Evaluation

The evaluation procedure is based on the AuctionNet simulator.

For evaluating the SEGB method, please run

python run/run_evaluate_segb.py

```