Inference endpoints and parallelism.

Merge branch 'main' of hf.co:spaces/meg/backend into main

Deleting logging that we're not using

Update src/backend/run_toxicity_eval.py

parallel processing handling

replicas

Background scheduling of the evaluation.

Handling more exceptions

Inference endpoint figuring

More handling of inference endpoints: Delete when done.

Trying to handle endpoint errors

Trying to handle endpoint errors

Adding more endpoint options

Updating with new approach to inference endpoint

Handling of json error, running generate all at once.

Please run after fully loading

Full dataset

Moving to just toxicity

Update src/envs.py

Changing to HF_TOKEN

Changes to whatever

Changing to BACKEND_TOKEN

Toxigen

string parsing error

Fixing timing bug

changing max new tokens to None due to continuing warning on backend: Running generate_until requests: 0%| | 13/99442 [10:42<1539:30:14, 55.74s/it]Both (=2048) and (=302) seem to have been set. will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)

changing batch size to auto

evaluation with hf not hf-auto

Run full eval

DEBUG

DEBUG

just realtoxicityprompts

Limit to 20 now

debug

debug

debug

debug

Debug

debug

Debug

Changing refresh rate to 1 hr

Updates, not sure what -- left over from last night

Refreshing less

Changing model from hf-causal-experimental to hf-auto

Trying to make it work with new EAI versions

Should be realtoxicityprompts as one of the target tasks

Removing argument throwing a TypeError

Update src/backend/run_eval_suite_harness.py

Remove parens

Handling change in name of ALL_TASKS