2024-11-18

To Ask Israel

• Grant requirements
• Problems with frameworks (OOM etc?)
• Constraints enforced?
• Datasets tested?
• How did you set it up
• Snellius or something else?

Info

OpenMLSuites:
Sparse datasets = turn off inference time evaluation (inference_time_measurements)
tasks_ids = [360932, 360933]AMLB:
Increase time overhead_time_multiplier or overhead_time_multiplier
inference_time_measurements = 5? (do it to 10) Fix versions of the frameworksFrameworks

• flaml:

If there is a problem with “pandas”
Only install it inside the virtual env of flaml
If there is a problem with “callback”:
Only install the next inside the virtual env of flaml
pip install xgboost==2.0.3

• GAMA:

a package from sklearn should be updated (I don’t remember)

• H2OAutoML:

Define the memory manually (exec.py)jvm_memory = str(round(config.max_mem_size_mb * 2/3))+"M"

• NaiveAutoML:
• FEDOT:
• AutoGluon

“NoResultError: Trainer has no fit models that can infer.” (Figure)
Slurm:
AutoGluon does not respect the resources defined.
Run each fold per each framework:
python runbenchmark.py AutoGluon_benchmark:2024Q1 openml/t/360975 5min8c_gp3 -f 0
Clean cache
find -O3 /gpfs/scratch1/nodespecific/ -maxdepth 2 -type d -user icampero -exec rm -rf {} \;
Touch:
find israelAutoML2 -exec touch -a -m {} \;  (edited)

• https://apptainer.org/

Slurm

• myquota
• rome/thin or genoa
• run it with 28GB memory
  • problem with h20 and autogloun
• global save : false, retrieve save from backup
• clean cache
  • inode
  • 2 weeks without touching folder
  • use scratch to run your stuff