# BIG-bench SeqIO Tasks and Mixtures Catalog


## BIG-bench Lite SeqIO Mixtures


SeqIO Mixture | BIG-bench Tasks
----|----
`bigbench:bigbench_lite_v1.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:bigbench_lite_v1.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:bigbench_lite_v1.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:bigbench_lite_v1.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:bigbench_lite_v1.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:bigbench_lite_v1.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:bigbench_lite_v1.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:bigbench_lite_v1.mix.t5_default_vocab.3_shot.all_examples` | [auto_debugging](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/auto_debugging/)<br>[bbq_lite_json](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/bbq_lite_json/)<br>[code_line_description](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/code_line_description/)<br>[conceptual_combinations](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/conceptual_combinations/)<br>[conlang_translation](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/conlang_translation/)<br>[emoji_movie](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/emoji_movie/)<br>[formal_fallacies_syllogisms_negation](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/formal_fallacies_syllogisms_negation/)<br>[hindu_knowledge](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/hindu_knowledge/)<br>[known_unknowns](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/known_unknowns/)<br>[language_identification](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/language_identification/)<br>[linguistics_puzzles](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/linguistics_puzzles/)<br>[logic_grid_puzzle](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/logic_grid_puzzle/)<br>[logical_deduction](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/logical_deduction/)<br>[misconceptions_russian](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/misconceptions_russian/)<br>[novel_concepts](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/novel_concepts/)<br>[operators](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/operators/)<br>[parsinlu_reading_comprehension](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/parsinlu_reading_comprehension/)<br>[play_dialog_same_or_different](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/play_dialog_same_or_different/)<br>[repeat_copy_logic](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/repeat_copy_logic/)<br>[strange_stories](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/strange_stories/)<br>[strategyqa](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/strategyqa/)<br>[symbol_interpretation](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/symbol_interpretation/)<br>[vitaminc_fact_verification](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/vitaminc_fact_verification/)<br>[winowhy](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/winowhy/)



## All BIG-bench SeqIO Tasks


BIG-bench Task | SeqIO Tasks | Subtasks | Description | Type
----|----|----|----|----
[anachronisms](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/anachronisms/) | <details><summary>Tasks</summary><br>`bigbench:anachronisms.mul.t5_default_vocab.0_shot.230_examples`<br>`bigbench:anachronisms.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:anachronisms.mul.t5_default_vocab.1_shot.230_examples`<br>`bigbench:anachronisms.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:anachronisms.mul.t5_default_vocab.2_shot.230_examples`<br>`bigbench:anachronisms.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:anachronisms.mul.t5_default_vocab.3_shot.230_examples`<br>`bigbench:anachronisms.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify whether a given statement contains an anachronism</details> | Multiple-Choice
[analogical_similarity](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/analogical_similarity/) | <details><summary>Tasks</summary><br>`bigbench:analogical_similarity.mul.t5_default_vocab.0_shot.323_examples`<br>`bigbench:analogical_similarity.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:analogical_similarity.mul.t5_default_vocab.1_shot.323_examples`<br>`bigbench:analogical_similarity.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:analogical_similarity.mul.t5_default_vocab.2_shot.323_examples`<br>`bigbench:analogical_similarity.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:analogical_similarity.mul.t5_default_vocab.3_shot.323_examples`<br>`bigbench:analogical_similarity.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify the type of analogy between two events</details> | Multiple-Choice
[analytic_entailment](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/analytic_entailment/) | <details><summary>Tasks</summary><br>`bigbench:analytic_entailment.mul.t5_default_vocab.0_shot.70_examples`<br>`bigbench:analytic_entailment.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:analytic_entailment.mul.t5_default_vocab.1_shot.70_examples`<br>`bigbench:analytic_entailment.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:analytic_entailment.mul.t5_default_vocab.2_shot.70_examples`<br>`bigbench:analytic_entailment.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:analytic_entailment.mul.t5_default_vocab.3_shot.70_examples`<br>`bigbench:analytic_entailment.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify whether one sentence entails the next</details> | Multiple-Choice
[ascii_word_recognition](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/ascii_word_recognition/) | <details><summary>Tasks</summary><br>`bigbench:ascii_word_recognition.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:ascii_word_recognition.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:ascii_word_recognition.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:ascii_word_recognition.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:ascii_word_recognition.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:ascii_word_recognition.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:ascii_word_recognition.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:ascii_word_recognition.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify the word displayed as ASCII art</details> | Generative, Scoring
[auto_categorization](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/auto_categorization/) | <details><summary>Tasks</summary><br>`bigbench:auto_categorization.mix.t5_default_vocab.0_shot.328_examples`<br>`bigbench:auto_categorization.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:auto_categorization.mix.t5_default_vocab.1_shot.328_examples`<br>`bigbench:auto_categorization.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:auto_categorization.mix.t5_default_vocab.2_shot.328_examples`<br>`bigbench:auto_categorization.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:auto_categorization.mix.t5_default_vocab.3_shot.328_examples`<br>`bigbench:auto_categorization.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify a broad class given several examples from that class</details> | Generative, Scoring
[auto_debugging](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/auto_debugging/) | <details><summary>Tasks</summary><br>`bigbench:auto_debugging.mix.t5_default_vocab.0_shot.34_examples`<br>`bigbench:auto_debugging.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:auto_debugging.mix.t5_default_vocab.1_shot.34_examples`<br>`bigbench:auto_debugging.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:auto_debugging.mix.t5_default_vocab.2_shot.34_examples`<br>`bigbench:auto_debugging.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:auto_debugging.mix.t5_default_vocab.3_shot.34_examples`<br>`bigbench:auto_debugging.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer questions about a computer program's intermediate state</details> | Generative, Scoring
[bridging_anaphora_resolution_barqa](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/bridging_anaphora_resolution_barqa/) | <details><summary>Tasks</summary><br>`bigbench:bridging_anaphora_resolution_barqa.mix.t5_default_vocab.0_shot.648_examples`<br>`bigbench:bridging_anaphora_resolution_barqa.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:bridging_anaphora_resolution_barqa.mix.t5_default_vocab.1_shot.648_examples`<br>`bigbench:bridging_anaphora_resolution_barqa.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:bridging_anaphora_resolution_barqa.mix.t5_default_vocab.2_shot.648_examples`<br>`bigbench:bridging_anaphora_resolution_barqa.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:bridging_anaphora_resolution_barqa.mix.t5_default_vocab.3_shot.648_examples`<br>`bigbench:bridging_anaphora_resolution_barqa.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>An indirect anaphora resolution task that is casted as a context dependent question answering problem</details> | Generative, Scoring
[causal_judgment](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/causal_judgment/) | <details><summary>Tasks</summary><br>`bigbench:causal_judgment.mul.t5_default_vocab.0_shot.190_examples`<br>`bigbench:causal_judgment.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:causal_judgment.mul.t5_default_vocab.1_shot.190_examples`<br>`bigbench:causal_judgment.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:causal_judgment.mul.t5_default_vocab.2_shot.190_examples`<br>`bigbench:causal_judgment.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:causal_judgment.mul.t5_default_vocab.3_shot.190_examples`<br>`bigbench:causal_judgment.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer questions about causal attribution</details> | Multiple-Choice
[checkmate_in_one](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/checkmate_in_one/) | <details><summary>Tasks</summary><br>`bigbench:checkmate_in_one.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:checkmate_in_one.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:checkmate_in_one.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:checkmate_in_one.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:checkmate_in_one.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:checkmate_in_one.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:checkmate_in_one.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:checkmate_in_one.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Find a move in the chess position resulting in checkmate</details> | Generative, Multiple-Choice, Scoring
[chinese_remainder_theorem](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/chinese_remainder_theorem/) | <details><summary>Tasks</summary><br>`bigbench:chinese_remainder_theorem.mix.t5_default_vocab.0_shot.500_examples`<br>`bigbench:chinese_remainder_theorem.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:chinese_remainder_theorem.mix.t5_default_vocab.1_shot.500_examples`<br>`bigbench:chinese_remainder_theorem.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:chinese_remainder_theorem.mix.t5_default_vocab.2_shot.500_examples`<br>`bigbench:chinese_remainder_theorem.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:chinese_remainder_theorem.mix.t5_default_vocab.3_shot.500_examples`<br>`bigbench:chinese_remainder_theorem.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Solve basic number theory problems generated by the Chinese remainder theorem</details> | Generative, Scoring
[code_line_description](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/code_line_description/) | <details><summary>Tasks</summary><br>`bigbench:code_line_description.mul.t5_default_vocab.0_shot.60_examples`<br>`bigbench:code_line_description.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:code_line_description.mul.t5_default_vocab.1_shot.60_examples`<br>`bigbench:code_line_description.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:code_line_description.mul.t5_default_vocab.2_shot.60_examples`<br>`bigbench:code_line_description.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:code_line_description.mul.t5_default_vocab.3_shot.60_examples`<br>`bigbench:code_line_description.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Give an English language description of Python code</details> | Multiple-Choice
[codenames](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/codenames/) | <details><summary>Tasks</summary><br>`bigbench:codenames.mix.t5_default_vocab.0_shot.85_examples`<br>`bigbench:codenames.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:codenames.mix.t5_default_vocab.1_shot.85_examples`<br>`bigbench:codenames.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:codenames.mix.t5_default_vocab.2_shot.85_examples`<br>`bigbench:codenames.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:codenames.mix.t5_default_vocab.3_shot.85_examples`<br>`bigbench:codenames.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify words associated with a given word</details> | Generative, Scoring
[common_morpheme](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/common_morpheme/) | <details><summary>Tasks</summary><br>`bigbench:common_morpheme.mul.t5_default_vocab.0_shot.50_examples`<br>`bigbench:common_morpheme.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:common_morpheme.mul.t5_default_vocab.1_shot.50_examples`<br>`bigbench:common_morpheme.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:common_morpheme.mul.t5_default_vocab.2_shot.50_examples`<br>`bigbench:common_morpheme.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:common_morpheme.mul.t5_default_vocab.3_shot.50_examples`<br>`bigbench:common_morpheme.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Determine the meaning of the shared morpheme among the given words</details> | Multiple-Choice
[contextual_parametric_knowledge_conflicts](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/contextual_parametric_knowledge_conflicts/) | <details><summary>Tasks</summary><br>`bigbench:contextual_parametric_knowledge_conflicts.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:contextual_parametric_knowledge_conflicts.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:contextual_parametric_knowledge_conflicts.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:contextual_parametric_knowledge_conflicts.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:contextual_parametric_knowledge_conflicts.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:contextual_parametric_knowledge_conflicts.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:contextual_parametric_knowledge_conflicts.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:contextual_parametric_knowledge_conflicts.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer questions given the contextual information, though it may conflict with memorized, parametric knowledge.</details> | Generative, Multiple-Choice, Scoring
[crash_blossom](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/crash_blossom/) | <details><summary>Tasks</summary><br>`bigbench:crash_blossom.mul.t5_default_vocab.0_shot.38_examples`<br>`bigbench:crash_blossom.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:crash_blossom.mul.t5_default_vocab.1_shot.38_examples`<br>`bigbench:crash_blossom.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:crash_blossom.mul.t5_default_vocab.2_shot.38_examples`<br>`bigbench:crash_blossom.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:crash_blossom.mul.t5_default_vocab.3_shot.38_examples`<br>`bigbench:crash_blossom.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Disambiguate the part-of-speech of ambiguous words or phrases</details> | Multiple-Choice
[crass_ai](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/crass_ai/) | <details><summary>Tasks</summary><br>`bigbench:crass_ai.mul.t5_default_vocab.0_shot.44_examples`<br>`bigbench:crass_ai.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:crass_ai.mul.t5_default_vocab.1_shot.44_examples`<br>`bigbench:crass_ai.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:crass_ai.mul.t5_default_vocab.2_shot.44_examples`<br>`bigbench:crass_ai.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:crass_ai.mul.t5_default_vocab.3_shot.44_examples`<br>`bigbench:crass_ai.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Predict effects of causal events in counterfactual situations</details> | Multiple-Choice
[cryobiology_spanish](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/cryobiology_spanish/) | <details><summary>Tasks</summary><br>`bigbench:cryobiology_spanish.mul.t5_default_vocab.0_shot.146_examples`<br>`bigbench:cryobiology_spanish.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:cryobiology_spanish.mul.t5_default_vocab.1_shot.146_examples`<br>`bigbench:cryobiology_spanish.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:cryobiology_spanish.mul.t5_default_vocab.2_shot.146_examples`<br>`bigbench:cryobiology_spanish.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:cryobiology_spanish.mul.t5_default_vocab.3_shot.146_examples`<br>`bigbench:cryobiology_spanish.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer questions (in Spanish) about cryobiology</details> | Multiple-Choice
[cryptonite](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/cryptonite/) | <details><summary>Tasks</summary><br>`bigbench:cryptonite.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:cryptonite.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:cryptonite.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:cryptonite.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:cryptonite.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:cryptonite.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:cryptonite.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:cryptonite.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Solve the cryptic crossword clues</details> | Generative, Scoring
[dark_humor_detection](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/dark_humor_detection/) | <details><summary>Tasks</summary><br>`bigbench:dark_humor_detection.mul.t5_default_vocab.0_shot.80_examples`<br>`bigbench:dark_humor_detection.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:dark_humor_detection.mul.t5_default_vocab.1_shot.80_examples`<br>`bigbench:dark_humor_detection.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:dark_humor_detection.mul.t5_default_vocab.2_shot.80_examples`<br>`bigbench:dark_humor_detection.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:dark_humor_detection.mul.t5_default_vocab.3_shot.80_examples`<br>`bigbench:dark_humor_detection.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Determine if the given text is intended to be a joke (with dark humor) or not</details> | Multiple-Choice
[date_understanding](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/date_understanding/) | <details><summary>Tasks</summary><br>`bigbench:date_understanding.mul.t5_default_vocab.0_shot.369_examples`<br>`bigbench:date_understanding.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:date_understanding.mul.t5_default_vocab.1_shot.369_examples`<br>`bigbench:date_understanding.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:date_understanding.mul.t5_default_vocab.2_shot.369_examples`<br>`bigbench:date_understanding.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:date_understanding.mul.t5_default_vocab.3_shot.369_examples`<br>`bigbench:date_understanding.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Infer the date from context</details> | Multiple-Choice
[disambiguation_qa](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/disambiguation_qa/) | <details><summary>Tasks</summary><br>`bigbench:disambiguation_qa.mul.t5_default_vocab.0_shot.258_examples`<br>`bigbench:disambiguation_qa.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:disambiguation_qa.mul.t5_default_vocab.1_shot.258_examples`<br>`bigbench:disambiguation_qa.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:disambiguation_qa.mul.t5_default_vocab.2_shot.258_examples`<br>`bigbench:disambiguation_qa.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:disambiguation_qa.mul.t5_default_vocab.3_shot.258_examples`<br>`bigbench:disambiguation_qa.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Clarify the meaning of sentences with ambiguous pronouns</details> | Multiple-Choice
[discourse_marker_prediction](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/discourse_marker_prediction/) | <details><summary>Tasks</summary><br>`bigbench:discourse_marker_prediction.mul.t5_default_vocab.0_shot.857_examples`<br>`bigbench:discourse_marker_prediction.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:discourse_marker_prediction.mul.t5_default_vocab.1_shot.857_examples`<br>`bigbench:discourse_marker_prediction.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:discourse_marker_prediction.mul.t5_default_vocab.2_shot.857_examples`<br>`bigbench:discourse_marker_prediction.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:discourse_marker_prediction.mul.t5_default_vocab.3_shot.857_examples`<br>`bigbench:discourse_marker_prediction.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Predict the discourse marker continuation</details> | Multiple-Choice
[disfl_qa](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/disfl_qa/) | <details><summary>Tasks</summary><br>`bigbench:disfl_qa.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:disfl_qa.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:disfl_qa.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:disfl_qa.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:disfl_qa.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:disfl_qa.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:disfl_qa.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:disfl_qa.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Pick the correct answer span from the context given the disfluent question</details> | Generative, Scoring
[dyck_languages](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/dyck_languages/) | <details><summary>Tasks</summary><br>`bigbench:dyck_languages.mul.t5_default_vocab.0_shot.1000_examples`<br>`bigbench:dyck_languages.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:dyck_languages.mul.t5_default_vocab.1_shot.1000_examples`<br>`bigbench:dyck_languages.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:dyck_languages.mul.t5_default_vocab.2_shot.1000_examples`<br>`bigbench:dyck_languages.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:dyck_languages.mul.t5_default_vocab.3_shot.1000_examples`<br>`bigbench:dyck_languages.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Correctly close a Dyck-n word</details> | Multiple-Choice
[emoji_movie](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/emoji_movie/) | <details><summary>Tasks</summary><br>`bigbench:emoji_movie.mix.t5_default_vocab.0_shot.100_examples`<br>`bigbench:emoji_movie.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:emoji_movie.mix.t5_default_vocab.1_shot.100_examples`<br>`bigbench:emoji_movie.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:emoji_movie.mix.t5_default_vocab.2_shot.100_examples`<br>`bigbench:emoji_movie.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:emoji_movie.mix.t5_default_vocab.3_shot.100_examples`<br>`bigbench:emoji_movie.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Guess popular movies from their emoji descriptions</details> | Generative, Multiple-Choice, Scoring
[emojis_emotion_prediction](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/emojis_emotion_prediction/) | <details><summary>Tasks</summary><br>`bigbench:emojis_emotion_prediction.mul.t5_default_vocab.0_shot.131_examples`<br>`bigbench:emojis_emotion_prediction.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:emojis_emotion_prediction.mul.t5_default_vocab.1_shot.131_examples`<br>`bigbench:emojis_emotion_prediction.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:emojis_emotion_prediction.mul.t5_default_vocab.2_shot.131_examples`<br>`bigbench:emojis_emotion_prediction.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:emojis_emotion_prediction.mul.t5_default_vocab.3_shot.131_examples`<br>`bigbench:emojis_emotion_prediction.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Predict the emotion of a given emoji</details> | Multiple-Choice
[empirical_judgments](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/empirical_judgments/) | <details><summary>Tasks</summary><br>`bigbench:empirical_judgments.mul.t5_default_vocab.0_shot.99_examples`<br>`bigbench:empirical_judgments.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:empirical_judgments.mul.t5_default_vocab.1_shot.99_examples`<br>`bigbench:empirical_judgments.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:empirical_judgments.mul.t5_default_vocab.2_shot.99_examples`<br>`bigbench:empirical_judgments.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:empirical_judgments.mul.t5_default_vocab.3_shot.99_examples`<br>`bigbench:empirical_judgments.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Distinguish between causal and correlative empirical judgements</details> | Multiple-Choice
[english_proverbs](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/english_proverbs/) | <details><summary>Tasks</summary><br>`bigbench:english_proverbs.mul.t5_default_vocab.0_shot.34_examples`<br>`bigbench:english_proverbs.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:english_proverbs.mul.t5_default_vocab.1_shot.34_examples`<br>`bigbench:english_proverbs.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:english_proverbs.mul.t5_default_vocab.2_shot.34_examples`<br>`bigbench:english_proverbs.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:english_proverbs.mul.t5_default_vocab.3_shot.34_examples`<br>`bigbench:english_proverbs.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Find the English proverb corresponding to the given story</details> | Multiple-Choice
[english_russian_proverbs](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/english_russian_proverbs/) | <details><summary>Tasks</summary><br>`bigbench:english_russian_proverbs.mul.t5_default_vocab.0_shot.80_examples`<br>`bigbench:english_russian_proverbs.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:english_russian_proverbs.mul.t5_default_vocab.1_shot.80_examples`<br>`bigbench:english_russian_proverbs.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:english_russian_proverbs.mul.t5_default_vocab.2_shot.80_examples`<br>`bigbench:english_russian_proverbs.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:english_russian_proverbs.mul.t5_default_vocab.3_shot.80_examples`<br>`bigbench:english_russian_proverbs.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>For a given proverb in English, choose a proverb in Russian which is closest in meaning</details> | Multiple-Choice
[entailed_polarity](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/entailed_polarity/) | <details><summary>Tasks</summary><br>`bigbench:entailed_polarity.mul.t5_default_vocab.0_shot.148_examples`<br>`bigbench:entailed_polarity.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:entailed_polarity.mul.t5_default_vocab.1_shot.148_examples`<br>`bigbench:entailed_polarity.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:entailed_polarity.mul.t5_default_vocab.2_shot.148_examples`<br>`bigbench:entailed_polarity.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:entailed_polarity.mul.t5_default_vocab.3_shot.148_examples`<br>`bigbench:entailed_polarity.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Infer the entailed polarity</details> | Multiple-Choice
[entailed_polarity_hindi](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/entailed_polarity_hindi/) | <details><summary>Tasks</summary><br>`bigbench:entailed_polarity_hindi.mul.t5_default_vocab.0_shot.138_examples`<br>`bigbench:entailed_polarity_hindi.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:entailed_polarity_hindi.mul.t5_default_vocab.1_shot.138_examples`<br>`bigbench:entailed_polarity_hindi.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:entailed_polarity_hindi.mul.t5_default_vocab.2_shot.138_examples`<br>`bigbench:entailed_polarity_hindi.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:entailed_polarity_hindi.mul.t5_default_vocab.3_shot.138_examples`<br>`bigbench:entailed_polarity_hindi.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Infer the entailed polarity (Hindi)</details> | Multiple-Choice
[epistemic_reasoning](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/epistemic_reasoning/) | <details><summary>Tasks</summary><br>`bigbench:epistemic_reasoning.mul.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:epistemic_reasoning.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:epistemic_reasoning.mul.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:epistemic_reasoning.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:epistemic_reasoning.mul.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:epistemic_reasoning.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:epistemic_reasoning.mul.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:epistemic_reasoning.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Determine whether one sentence entails the next</details> | Multiple-Choice
[evaluating_information_essentiality](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/evaluating_information_essentiality/) | <details><summary>Tasks</summary><br>`bigbench:evaluating_information_essentiality.mul.t5_default_vocab.0_shot.68_examples`<br>`bigbench:evaluating_information_essentiality.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:evaluating_information_essentiality.mul.t5_default_vocab.1_shot.68_examples`<br>`bigbench:evaluating_information_essentiality.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:evaluating_information_essentiality.mul.t5_default_vocab.2_shot.68_examples`<br>`bigbench:evaluating_information_essentiality.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:evaluating_information_essentiality.mul.t5_default_vocab.3_shot.68_examples`<br>`bigbench:evaluating_information_essentiality.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify statements that are essential to answer a question</details> | Multiple-Choice
[fantasy_reasoning](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/fantasy_reasoning/) | <details><summary>Tasks</summary><br>`bigbench:fantasy_reasoning.mul.t5_default_vocab.0_shot.201_examples`<br>`bigbench:fantasy_reasoning.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:fantasy_reasoning.mul.t5_default_vocab.1_shot.201_examples`<br>`bigbench:fantasy_reasoning.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:fantasy_reasoning.mul.t5_default_vocab.2_shot.201_examples`<br>`bigbench:fantasy_reasoning.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:fantasy_reasoning.mul.t5_default_vocab.3_shot.201_examples`<br>`bigbench:fantasy_reasoning.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Reason in a world where common sense does not apply</details> | Multiple-Choice
[few_shot_nlg](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/few_shot_nlg/) | <details><summary>Tasks</summary><br>`bigbench:few_shot_nlg.mix.t5_default_vocab.0_shot.153_examples`<br>`bigbench:few_shot_nlg.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:few_shot_nlg.mix.t5_default_vocab.1_shot.153_examples`<br>`bigbench:few_shot_nlg.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:few_shot_nlg.mix.t5_default_vocab.2_shot.153_examples`<br>`bigbench:few_shot_nlg.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:few_shot_nlg.mix.t5_default_vocab.3_shot.153_examples`<br>`bigbench:few_shot_nlg.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Generate natural language from structured data in a few-shot setup</details> | Generative, Scoring
[figure_of_speech_detection](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/figure_of_speech_detection/) | <details><summary>Tasks</summary><br>`bigbench:figure_of_speech_detection.mul.t5_default_vocab.0_shot.59_examples`<br>`bigbench:figure_of_speech_detection.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:figure_of_speech_detection.mul.t5_default_vocab.1_shot.59_examples`<br>`bigbench:figure_of_speech_detection.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:figure_of_speech_detection.mul.t5_default_vocab.2_shot.59_examples`<br>`bigbench:figure_of_speech_detection.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:figure_of_speech_detection.mul.t5_default_vocab.3_shot.59_examples`<br>`bigbench:figure_of_speech_detection.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify the figure of speech embodied by the sentence</details> | Multiple-Choice
[formal_fallacies_syllogisms_negation](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/formal_fallacies_syllogisms_negation/) | <details><summary>Tasks</summary><br>`bigbench:formal_fallacies_syllogisms_negation.mul.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:formal_fallacies_syllogisms_negation.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:formal_fallacies_syllogisms_negation.mul.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:formal_fallacies_syllogisms_negation.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:formal_fallacies_syllogisms_negation.mul.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:formal_fallacies_syllogisms_negation.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:formal_fallacies_syllogisms_negation.mul.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:formal_fallacies_syllogisms_negation.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Distinguish deductively valid arguments from formal fallacies</details> | Multiple-Choice
[gender_inclusive_sentences_german](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/gender_inclusive_sentences_german/) | <details><summary>Tasks</summary><br>`bigbench:gender_inclusive_sentences_german.mix.t5_default_vocab.0_shot.200_examples`<br>`bigbench:gender_inclusive_sentences_german.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:gender_inclusive_sentences_german.mix.t5_default_vocab.1_shot.200_examples`<br>`bigbench:gender_inclusive_sentences_german.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:gender_inclusive_sentences_german.mix.t5_default_vocab.2_shot.200_examples`<br>`bigbench:gender_inclusive_sentences_german.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:gender_inclusive_sentences_german.mix.t5_default_vocab.3_shot.200_examples`<br>`bigbench:gender_inclusive_sentences_german.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Given a German language sentence that does not use gender-inclusive forms, transform it so that it uses gender-inclusive forms using the '*' character or other gender-neutral terms</details> | Generative, Scoring
[general_knowledge](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/general_knowledge/) | <details><summary>Tasks</summary><br>`bigbench:general_knowledge.mul.t5_default_vocab.0_shot.70_examples`<br>`bigbench:general_knowledge.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:general_knowledge.mul.t5_default_vocab.1_shot.70_examples`<br>`bigbench:general_knowledge.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:general_knowledge.mul.t5_default_vocab.2_shot.70_examples`<br>`bigbench:general_knowledge.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:general_knowledge.mul.t5_default_vocab.3_shot.70_examples`<br>`bigbench:general_knowledge.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer basic general-knowledge questions</details> | Multiple-Choice
[geometric_shapes](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/geometric_shapes/) | <details><summary>Tasks</summary><br>`bigbench:geometric_shapes.mix.t5_default_vocab.0_shot.359_examples`<br>`bigbench:geometric_shapes.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:geometric_shapes.mix.t5_default_vocab.1_shot.359_examples`<br>`bigbench:geometric_shapes.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:geometric_shapes.mix.t5_default_vocab.2_shot.359_examples`<br>`bigbench:geometric_shapes.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:geometric_shapes.mix.t5_default_vocab.3_shot.359_examples`<br>`bigbench:geometric_shapes.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Name geometric shapes from their SVG paths</details> | Generative, Multiple-Choice, Scoring
[gre_reading_comprehension](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/gre_reading_comprehension/) | <details><summary>Tasks</summary><br>`bigbench:gre_reading_comprehension.mul.t5_default_vocab.0_shot.31_examples`<br>`bigbench:gre_reading_comprehension.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:gre_reading_comprehension.mul.t5_default_vocab.1_shot.31_examples`<br>`bigbench:gre_reading_comprehension.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:gre_reading_comprehension.mul.t5_default_vocab.2_shot.31_examples`<br>`bigbench:gre_reading_comprehension.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:gre_reading_comprehension.mul.t5_default_vocab.3_shot.31_examples`<br>`bigbench:gre_reading_comprehension.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Given a passage from a GRE practice test and a question, find the best fitting answer</details> | Multiple-Choice
[hindi_question_answering](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/hindi_question_answering/) | <details><summary>Tasks</summary><br>`bigbench:hindi_question_answering.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:hindi_question_answering.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:hindi_question_answering.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:hindi_question_answering.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:hindi_question_answering.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:hindi_question_answering.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:hindi_question_answering.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:hindi_question_answering.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer questions in Hindi</details> | Generative, Scoring
[hindu_knowledge](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/hindu_knowledge/) | <details><summary>Tasks</summary><br>`bigbench:hindu_knowledge.mul.t5_default_vocab.0_shot.175_examples`<br>`bigbench:hindu_knowledge.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:hindu_knowledge.mul.t5_default_vocab.1_shot.175_examples`<br>`bigbench:hindu_knowledge.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:hindu_knowledge.mul.t5_default_vocab.2_shot.175_examples`<br>`bigbench:hindu_knowledge.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:hindu_knowledge.mul.t5_default_vocab.3_shot.175_examples`<br>`bigbench:hindu_knowledge.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer questions about Hindu mythology</details> | Multiple-Choice
[hinglish_toxicity](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/hinglish_toxicity/) | <details><summary>Tasks</summary><br>`bigbench:hinglish_toxicity.mul.t5_default_vocab.0_shot.200_examples`<br>`bigbench:hinglish_toxicity.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:hinglish_toxicity.mul.t5_default_vocab.1_shot.200_examples`<br>`bigbench:hinglish_toxicity.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:hinglish_toxicity.mul.t5_default_vocab.2_shot.200_examples`<br>`bigbench:hinglish_toxicity.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:hinglish_toxicity.mul.t5_default_vocab.3_shot.200_examples`<br>`bigbench:hinglish_toxicity.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Predict if a Hinglish sentence is toxic or not</details> | Multiple-Choice
[human_organs_senses](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/human_organs_senses/) | <details><summary>Tasks</summary><br>`bigbench:human_organs_senses.mul.t5_default_vocab.0_shot.42_examples`<br>`bigbench:human_organs_senses.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:human_organs_senses.mul.t5_default_vocab.1_shot.42_examples`<br>`bigbench:human_organs_senses.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:human_organs_senses.mul.t5_default_vocab.2_shot.42_examples`<br>`bigbench:human_organs_senses.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:human_organs_senses.mul.t5_default_vocab.3_shot.42_examples`<br>`bigbench:human_organs_senses.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer questions about human senses and organs</details> | Multiple-Choice
[hyperbaton](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/hyperbaton/) | <details><summary>Tasks</summary><br>`bigbench:hyperbaton.mul.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:hyperbaton.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:hyperbaton.mul.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:hyperbaton.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:hyperbaton.mul.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:hyperbaton.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:hyperbaton.mul.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:hyperbaton.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Order adjectives correctly in English sentences</details> | Multiple-Choice
[identify_math_theorems](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/identify_math_theorems/) | <details><summary>Tasks</summary><br>`bigbench:identify_math_theorems.mul.t5_default_vocab.0_shot.53_examples`<br>`bigbench:identify_math_theorems.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:identify_math_theorems.mul.t5_default_vocab.1_shot.53_examples`<br>`bigbench:identify_math_theorems.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:identify_math_theorems.mul.t5_default_vocab.2_shot.53_examples`<br>`bigbench:identify_math_theorems.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:identify_math_theorems.mul.t5_default_vocab.3_shot.53_examples`<br>`bigbench:identify_math_theorems.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Determine the veracity of the mathematical theorem and correct it if false</details> | Multiple-Choice
[identify_odd_metaphor](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/identify_odd_metaphor/) | <details><summary>Tasks</summary><br>`bigbench:identify_odd_metaphor.mul.t5_default_vocab.0_shot.47_examples`<br>`bigbench:identify_odd_metaphor.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:identify_odd_metaphor.mul.t5_default_vocab.1_shot.47_examples`<br>`bigbench:identify_odd_metaphor.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:identify_odd_metaphor.mul.t5_default_vocab.2_shot.47_examples`<br>`bigbench:identify_odd_metaphor.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:identify_odd_metaphor.mul.t5_default_vocab.3_shot.47_examples`<br>`bigbench:identify_odd_metaphor.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Select the sentence where the metaphorical language used about a given topic could not be applied to another specified topic</details> | Multiple-Choice
[implicatures](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/implicatures/) | <details><summary>Tasks</summary><br>`bigbench:implicatures.mul.t5_default_vocab.0_shot.492_examples`<br>`bigbench:implicatures.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:implicatures.mul.t5_default_vocab.1_shot.492_examples`<br>`bigbench:implicatures.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:implicatures.mul.t5_default_vocab.2_shot.492_examples`<br>`bigbench:implicatures.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:implicatures.mul.t5_default_vocab.3_shot.492_examples`<br>`bigbench:implicatures.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Predict whether Speaker 2's answer to Speaker 1 counts as a yes or as a no</details> | Multiple-Choice
[implicit_relations](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/implicit_relations/) | <details><summary>Tasks</summary><br>`bigbench:implicit_relations.mul.t5_default_vocab.0_shot.85_examples`<br>`bigbench:implicit_relations.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:implicit_relations.mul.t5_default_vocab.1_shot.85_examples`<br>`bigbench:implicit_relations.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:implicit_relations.mul.t5_default_vocab.2_shot.85_examples`<br>`bigbench:implicit_relations.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:implicit_relations.mul.t5_default_vocab.3_shot.85_examples`<br>`bigbench:implicit_relations.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Determine the relation between people described in context</details> | Multiple-Choice
[intent_recognition](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/intent_recognition/) | <details><summary>Tasks</summary><br>`bigbench:intent_recognition.mul.t5_default_vocab.0_shot.693_examples`<br>`bigbench:intent_recognition.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:intent_recognition.mul.t5_default_vocab.1_shot.693_examples`<br>`bigbench:intent_recognition.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:intent_recognition.mul.t5_default_vocab.2_shot.693_examples`<br>`bigbench:intent_recognition.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:intent_recognition.mul.t5_default_vocab.3_shot.693_examples`<br>`bigbench:intent_recognition.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Predict the intent of an utterance</details> | Multiple-Choice
[international_phonetic_alphabet_nli](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/international_phonetic_alphabet_nli/) | <details><summary>Tasks</summary><br>`bigbench:international_phonetic_alphabet_nli.mul.t5_default_vocab.0_shot.126_examples`<br>`bigbench:international_phonetic_alphabet_nli.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:international_phonetic_alphabet_nli.mul.t5_default_vocab.1_shot.126_examples`<br>`bigbench:international_phonetic_alphabet_nli.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:international_phonetic_alphabet_nli.mul.t5_default_vocab.2_shot.126_examples`<br>`bigbench:international_phonetic_alphabet_nli.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:international_phonetic_alphabet_nli.mul.t5_default_vocab.3_shot.126_examples`<br>`bigbench:international_phonetic_alphabet_nli.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Solve natural-language-inference tasks presented in the International Phonetic Alphabet (IPA)</details> | Multiple-Choice
[international_phonetic_alphabet_transliterate](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/international_phonetic_alphabet_transliterate/) | <details><summary>Tasks</summary><br>`bigbench:international_phonetic_alphabet_transliterate.mix.t5_default_vocab.0_shot.1003_examples`<br>`bigbench:international_phonetic_alphabet_transliterate.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:international_phonetic_alphabet_transliterate.mix.t5_default_vocab.1_shot.1003_examples`<br>`bigbench:international_phonetic_alphabet_transliterate.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:international_phonetic_alphabet_transliterate.mix.t5_default_vocab.2_shot.1003_examples`<br>`bigbench:international_phonetic_alphabet_transliterate.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:international_phonetic_alphabet_transliterate.mix.t5_default_vocab.3_shot.1003_examples`<br>`bigbench:international_phonetic_alphabet_transliterate.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Transliterate sentences between the International Phonetic Alphabet (IPA) and written English</details> | Generative, Scoring
[irony_identification](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/irony_identification/) | <details><summary>Tasks</summary><br>`bigbench:irony_identification.mul.t5_default_vocab.0_shot.99_examples`<br>`bigbench:irony_identification.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:irony_identification.mul.t5_default_vocab.1_shot.99_examples`<br>`bigbench:irony_identification.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:irony_identification.mul.t5_default_vocab.2_shot.99_examples`<br>`bigbench:irony_identification.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:irony_identification.mul.t5_default_vocab.3_shot.99_examples`<br>`bigbench:irony_identification.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify whether a given sentence/s is/are ironic or not</details> | Multiple-Choice
[kannada](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/kannada/) | <details><summary>Tasks</summary><br>`bigbench:kannada.mul.t5_default_vocab.0_shot.316_examples`<br>`bigbench:kannada.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:kannada.mul.t5_default_vocab.1_shot.316_examples`<br>`bigbench:kannada.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:kannada.mul.t5_default_vocab.2_shot.316_examples`<br>`bigbench:kannada.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:kannada.mul.t5_default_vocab.3_shot.316_examples`<br>`bigbench:kannada.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer Kannada riddles</details> | Multiple-Choice
[known_unknowns](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/known_unknowns/) | <details><summary>Tasks</summary><br>`bigbench:known_unknowns.mul.t5_default_vocab.0_shot.46_examples`<br>`bigbench:known_unknowns.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:known_unknowns.mul.t5_default_vocab.1_shot.46_examples`<br>`bigbench:known_unknowns.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:known_unknowns.mul.t5_default_vocab.2_shot.46_examples`<br>`bigbench:known_unknowns.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:known_unknowns.mul.t5_default_vocab.3_shot.46_examples`<br>`bigbench:known_unknowns.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>A test of 'hallucinations' by asking questions whose answers are known to be unknown</details> | Multiple-Choice
[language_identification](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/language_identification/) | <details><summary>Tasks</summary><br>`bigbench:language_identification.mul.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:language_identification.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:language_identification.mul.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:language_identification.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:language_identification.mul.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:language_identification.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:language_identification.mul.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:language_identification.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify the language a given sentence is written in</details> | Multiple-Choice
[linguistics_puzzles](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/linguistics_puzzles/) | <details><summary>Tasks</summary><br>`bigbench:linguistics_puzzles.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:linguistics_puzzles.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:linguistics_puzzles.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:linguistics_puzzles.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:linguistics_puzzles.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:linguistics_puzzles.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:linguistics_puzzles.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:linguistics_puzzles.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Solve Rosetta Stone-style linguistics puzzles</details> | Generative, Scoring
[logic_grid_puzzle](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/logic_grid_puzzle/) | <details><summary>Tasks</summary><br>`bigbench:logic_grid_puzzle.mul.t5_default_vocab.0_shot.1000_examples`<br>`bigbench:logic_grid_puzzle.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:logic_grid_puzzle.mul.t5_default_vocab.1_shot.1000_examples`<br>`bigbench:logic_grid_puzzle.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:logic_grid_puzzle.mul.t5_default_vocab.2_shot.1000_examples`<br>`bigbench:logic_grid_puzzle.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:logic_grid_puzzle.mul.t5_default_vocab.3_shot.1000_examples`<br>`bigbench:logic_grid_puzzle.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Solve logic grid puzzles</details> | Multiple-Choice
[logical_args](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/logical_args/) | <details><summary>Tasks</summary><br>`bigbench:logical_args.mul.t5_default_vocab.0_shot.32_examples`<br>`bigbench:logical_args.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:logical_args.mul.t5_default_vocab.1_shot.32_examples`<br>`bigbench:logical_args.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:logical_args.mul.t5_default_vocab.2_shot.32_examples`<br>`bigbench:logical_args.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:logical_args.mul.t5_default_vocab.3_shot.32_examples`<br>`bigbench:logical_args.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Find statements which strengthen or weaken logical arguments</details> | Multiple-Choice
[logical_fallacy_detection](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/logical_fallacy_detection/) | <details><summary>Tasks</summary><br>`bigbench:logical_fallacy_detection.mul.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:logical_fallacy_detection.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:logical_fallacy_detection.mul.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:logical_fallacy_detection.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:logical_fallacy_detection.mul.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:logical_fallacy_detection.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:logical_fallacy_detection.mul.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:logical_fallacy_detection.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Detect informal and formal logical fallacies</details> | Multiple-Choice
[logical_sequence](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/logical_sequence/) | <details><summary>Tasks</summary><br>`bigbench:logical_sequence.mul.t5_default_vocab.0_shot.39_examples`<br>`bigbench:logical_sequence.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:logical_sequence.mul.t5_default_vocab.1_shot.39_examples`<br>`bigbench:logical_sequence.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:logical_sequence.mul.t5_default_vocab.2_shot.39_examples`<br>`bigbench:logical_sequence.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:logical_sequence.mul.t5_default_vocab.3_shot.39_examples`<br>`bigbench:logical_sequence.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify the correct chronological or sequential order of items in a list</details> | Multiple-Choice
[mathematical_induction](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/mathematical_induction/) | <details><summary>Tasks</summary><br>`bigbench:mathematical_induction.mul.t5_default_vocab.0_shot.69_examples`<br>`bigbench:mathematical_induction.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:mathematical_induction.mul.t5_default_vocab.1_shot.69_examples`<br>`bigbench:mathematical_induction.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:mathematical_induction.mul.t5_default_vocab.2_shot.69_examples`<br>`bigbench:mathematical_induction.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:mathematical_induction.mul.t5_default_vocab.3_shot.69_examples`<br>`bigbench:mathematical_induction.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Verify mathematical induction proofs</details> | Multiple-Choice
[matrixshapes](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/matrixshapes/) | <details><summary>Tasks</summary><br>`bigbench:matrixshapes.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:matrixshapes.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:matrixshapes.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:matrixshapes.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:matrixshapes.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:matrixshapes.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:matrixshapes.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:matrixshapes.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Keep track of matrix shapes through various transformations</details> | Generative, Scoring
[metaphor_boolean](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/metaphor_boolean/) | <details><summary>Tasks</summary><br>`bigbench:metaphor_boolean.mul.t5_default_vocab.0_shot.680_examples`<br>`bigbench:metaphor_boolean.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:metaphor_boolean.mul.t5_default_vocab.1_shot.680_examples`<br>`bigbench:metaphor_boolean.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:metaphor_boolean.mul.t5_default_vocab.2_shot.680_examples`<br>`bigbench:metaphor_boolean.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:metaphor_boolean.mul.t5_default_vocab.3_shot.680_examples`<br>`bigbench:metaphor_boolean.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>For a given metaphoric sentence, identify if the second sentence is the correct interpretation</details> | Multiple-Choice
[misconceptions](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/misconceptions/) | <details><summary>Tasks</summary><br>`bigbench:misconceptions.mul.t5_default_vocab.0_shot.219_examples`<br>`bigbench:misconceptions.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:misconceptions.mul.t5_default_vocab.1_shot.219_examples`<br>`bigbench:misconceptions.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:misconceptions.mul.t5_default_vocab.2_shot.219_examples`<br>`bigbench:misconceptions.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:misconceptions.mul.t5_default_vocab.3_shot.219_examples`<br>`bigbench:misconceptions.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Distinguish true statements from common misconceptions.</details> | Multiple-Choice
[misconceptions_russian](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/misconceptions_russian/) | <details><summary>Tasks</summary><br>`bigbench:misconceptions_russian.mul.t5_default_vocab.0_shot.49_examples`<br>`bigbench:misconceptions_russian.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:misconceptions_russian.mul.t5_default_vocab.1_shot.49_examples`<br>`bigbench:misconceptions_russian.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:misconceptions_russian.mul.t5_default_vocab.2_shot.49_examples`<br>`bigbench:misconceptions_russian.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:misconceptions_russian.mul.t5_default_vocab.3_shot.49_examples`<br>`bigbench:misconceptions_russian.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify misconceptions in Russian</details> | Multiple-Choice
[mnist_ascii](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/mnist_ascii/) | <details><summary>Tasks</summary><br>`bigbench:mnist_ascii.mul.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:mnist_ascii.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:mnist_ascii.mul.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:mnist_ascii.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:mnist_ascii.mul.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:mnist_ascii.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:mnist_ascii.mul.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:mnist_ascii.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Classify MNIST Images converted to ASCII</details> | Multiple-Choice
[moral_permissibility](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/moral_permissibility/) | <details><summary>Tasks</summary><br>`bigbench:moral_permissibility.mul.t5_default_vocab.0_shot.342_examples`<br>`bigbench:moral_permissibility.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:moral_permissibility.mul.t5_default_vocab.1_shot.342_examples`<br>`bigbench:moral_permissibility.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:moral_permissibility.mul.t5_default_vocab.2_shot.342_examples`<br>`bigbench:moral_permissibility.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:moral_permissibility.mul.t5_default_vocab.3_shot.342_examples`<br>`bigbench:moral_permissibility.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Evaluate if AI makes moral permissibility judgments similar to human</details> | Multiple-Choice
[movie_dialog_same_or_different](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/movie_dialog_same_or_different/) | <details><summary>Tasks</summary><br>`bigbench:movie_dialog_same_or_different.mul.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:movie_dialog_same_or_different.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:movie_dialog_same_or_different.mul.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:movie_dialog_same_or_different.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:movie_dialog_same_or_different.mul.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:movie_dialog_same_or_different.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:movie_dialog_same_or_different.mul.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:movie_dialog_same_or_different.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Determine if adjacent movie conversation lines were spoken by the same individual</details> | Multiple-Choice
[movie_recommendation](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/movie_recommendation/) | <details><summary>Tasks</summary><br>`bigbench:movie_recommendation.mul.t5_default_vocab.0_shot.500_examples`<br>`bigbench:movie_recommendation.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:movie_recommendation.mul.t5_default_vocab.1_shot.500_examples`<br>`bigbench:movie_recommendation.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:movie_recommendation.mul.t5_default_vocab.2_shot.500_examples`<br>`bigbench:movie_recommendation.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:movie_recommendation.mul.t5_default_vocab.3_shot.500_examples`<br>`bigbench:movie_recommendation.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Recommend movies similar to the given list of movies</details> | Multiple-Choice
[navigate](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/navigate/) | <details><summary>Tasks</summary><br>`bigbench:navigate.mul.t5_default_vocab.0_shot.1000_examples`<br>`bigbench:navigate.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:navigate.mul.t5_default_vocab.1_shot.1000_examples`<br>`bigbench:navigate.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:navigate.mul.t5_default_vocab.2_shot.1000_examples`<br>`bigbench:navigate.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:navigate.mul.t5_default_vocab.3_shot.1000_examples`<br>`bigbench:navigate.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Given a series of navigation instructions, determine whether one would end up back at the starting point</details> | Multiple-Choice
[nonsense_words_grammar](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/nonsense_words_grammar/) | <details><summary>Tasks</summary><br>`bigbench:nonsense_words_grammar.mul.t5_default_vocab.0_shot.50_examples`<br>`bigbench:nonsense_words_grammar.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:nonsense_words_grammar.mul.t5_default_vocab.1_shot.50_examples`<br>`bigbench:nonsense_words_grammar.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:nonsense_words_grammar.mul.t5_default_vocab.2_shot.50_examples`<br>`bigbench:nonsense_words_grammar.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:nonsense_words_grammar.mul.t5_default_vocab.3_shot.50_examples`<br>`bigbench:nonsense_words_grammar.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Guess the grammatical role of new words</details> | Multiple-Choice
[novel_concepts](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/novel_concepts/) | <details><summary>Tasks</summary><br>`bigbench:novel_concepts.mul.t5_default_vocab.0_shot.32_examples`<br>`bigbench:novel_concepts.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:novel_concepts.mul.t5_default_vocab.1_shot.32_examples`<br>`bigbench:novel_concepts.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:novel_concepts.mul.t5_default_vocab.2_shot.32_examples`<br>`bigbench:novel_concepts.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:novel_concepts.mul.t5_default_vocab.3_shot.32_examples`<br>`bigbench:novel_concepts.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify what the given objects have in common</details> | Multiple-Choice
[object_counting](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/object_counting/) | <details><summary>Tasks</summary><br>`bigbench:object_counting.mix.t5_default_vocab.0_shot.1000_examples`<br>`bigbench:object_counting.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:object_counting.mix.t5_default_vocab.1_shot.1000_examples`<br>`bigbench:object_counting.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:object_counting.mix.t5_default_vocab.2_shot.1000_examples`<br>`bigbench:object_counting.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:object_counting.mix.t5_default_vocab.3_shot.1000_examples`<br>`bigbench:object_counting.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Questions that involve enumerating objects of different types and asking the model to count them</details> | Generative, Scoring
[odd_one_out](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/odd_one_out/) | <details><summary>Tasks</summary><br>`bigbench:odd_one_out.mul.t5_default_vocab.0_shot.86_examples`<br>`bigbench:odd_one_out.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:odd_one_out.mul.t5_default_vocab.1_shot.86_examples`<br>`bigbench:odd_one_out.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:odd_one_out.mul.t5_default_vocab.2_shot.86_examples`<br>`bigbench:odd_one_out.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:odd_one_out.mul.t5_default_vocab.3_shot.86_examples`<br>`bigbench:odd_one_out.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Spot the word that does not belong in the group (semantically or grammatically)</details> | Multiple-Choice
[operators](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/operators/) | <details><summary>Tasks</summary><br>`bigbench:operators.mix.t5_default_vocab.0_shot.210_examples`<br>`bigbench:operators.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:operators.mix.t5_default_vocab.1_shot.210_examples`<br>`bigbench:operators.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:operators.mix.t5_default_vocab.2_shot.210_examples`<br>`bigbench:operators.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:operators.mix.t5_default_vocab.3_shot.210_examples`<br>`bigbench:operators.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Given a mathematical operator definition in natural language, apply it</details> | Generative, Scoring
[paragraph_segmentation](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/paragraph_segmentation/) | <details><summary>Tasks</summary><br>`bigbench:paragraph_segmentation.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:paragraph_segmentation.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:paragraph_segmentation.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:paragraph_segmentation.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:paragraph_segmentation.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:paragraph_segmentation.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:paragraph_segmentation.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:paragraph_segmentation.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify the sentences that end a paragraph in a document</details> | Generative, Scoring
[parsinlu_qa](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/parsinlu_qa/) | <details><summary>Tasks</summary><br>`bigbench:parsinlu_qa.mul.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:parsinlu_qa.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:parsinlu_qa.mul.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:parsinlu_qa.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:parsinlu_qa.mul.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:parsinlu_qa.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:parsinlu_qa.mul.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:parsinlu_qa.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer multiple-choice questions in Persian</details> | Multiple-Choice
[parsinlu_reading_comprehension](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/parsinlu_reading_comprehension/) | <details><summary>Tasks</summary><br>`bigbench:parsinlu_reading_comprehension.mix.t5_default_vocab.0_shot.518_examples`<br>`bigbench:parsinlu_reading_comprehension.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:parsinlu_reading_comprehension.mix.t5_default_vocab.1_shot.518_examples`<br>`bigbench:parsinlu_reading_comprehension.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:parsinlu_reading_comprehension.mix.t5_default_vocab.2_shot.518_examples`<br>`bigbench:parsinlu_reading_comprehension.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:parsinlu_reading_comprehension.mix.t5_default_vocab.3_shot.518_examples`<br>`bigbench:parsinlu_reading_comprehension.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer reading comprehension questions from ParsiNLU, a suite of high-level NLP tasks for Persian language</details> | Generative, Scoring
[penguins_in_a_table](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/penguins_in_a_table/) | <details><summary>Tasks</summary><br>`bigbench:penguins_in_a_table.mix.t5_default_vocab.0_shot.149_examples`<br>`bigbench:penguins_in_a_table.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:penguins_in_a_table.mix.t5_default_vocab.1_shot.149_examples`<br>`bigbench:penguins_in_a_table.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:penguins_in_a_table.mix.t5_default_vocab.2_shot.149_examples`<br>`bigbench:penguins_in_a_table.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:penguins_in_a_table.mix.t5_default_vocab.3_shot.149_examples`<br>`bigbench:penguins_in_a_table.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer questions about a table of penguins and their attributes</details> | Generative, Multiple-Choice, Scoring
[persian_idioms](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/persian_idioms/) | <details><summary>Tasks</summary><br>`bigbench:persian_idioms.mul.t5_default_vocab.0_shot.66_examples`<br>`bigbench:persian_idioms.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:persian_idioms.mul.t5_default_vocab.1_shot.66_examples`<br>`bigbench:persian_idioms.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:persian_idioms.mul.t5_default_vocab.2_shot.66_examples`<br>`bigbench:persian_idioms.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:persian_idioms.mul.t5_default_vocab.3_shot.66_examples`<br>`bigbench:persian_idioms.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify the literal meaning of Persian idioms.</details> | Multiple-Choice
[phrase_relatedness](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/phrase_relatedness/) | <details><summary>Tasks</summary><br>`bigbench:phrase_relatedness.mul.t5_default_vocab.0_shot.100_examples`<br>`bigbench:phrase_relatedness.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:phrase_relatedness.mul.t5_default_vocab.1_shot.100_examples`<br>`bigbench:phrase_relatedness.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:phrase_relatedness.mul.t5_default_vocab.2_shot.100_examples`<br>`bigbench:phrase_relatedness.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:phrase_relatedness.mul.t5_default_vocab.3_shot.100_examples`<br>`bigbench:phrase_relatedness.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Given a phrase (n-gram), select the most related phrase (n-gram) among the choices</details> | Multiple-Choice
[physical_intuition](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/physical_intuition/) | <details><summary>Tasks</summary><br>`bigbench:physical_intuition.mul.t5_default_vocab.0_shot.81_examples`<br>`bigbench:physical_intuition.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:physical_intuition.mul.t5_default_vocab.1_shot.81_examples`<br>`bigbench:physical_intuition.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:physical_intuition.mul.t5_default_vocab.2_shot.81_examples`<br>`bigbench:physical_intuition.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:physical_intuition.mul.t5_default_vocab.3_shot.81_examples`<br>`bigbench:physical_intuition.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Deduce the physical mechanism or behavior associated with a physical system</details> | Multiple-Choice
[physics](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/physics/) | <details><summary>Tasks</summary><br>`bigbench:physics.mul.t5_default_vocab.0_shot.229_examples`<br>`bigbench:physics.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:physics.mul.t5_default_vocab.1_shot.229_examples`<br>`bigbench:physics.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:physics.mul.t5_default_vocab.2_shot.229_examples`<br>`bigbench:physics.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:physics.mul.t5_default_vocab.3_shot.229_examples`<br>`bigbench:physics.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify the formula required to solve a physics word problem</details> | Multiple-Choice
[physics_questions](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/physics_questions/) | <details><summary>Tasks</summary><br>`bigbench:physics_questions.mix.t5_default_vocab.0_shot.54_examples`<br>`bigbench:physics_questions.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:physics_questions.mix.t5_default_vocab.1_shot.54_examples`<br>`bigbench:physics_questions.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:physics_questions.mix.t5_default_vocab.2_shot.54_examples`<br>`bigbench:physics_questions.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:physics_questions.mix.t5_default_vocab.3_shot.54_examples`<br>`bigbench:physics_questions.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer high-school-level physics multiple-choice questions</details> | Generative, Scoring
[play_dialog_same_or_different](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/play_dialog_same_or_different/) | <details><summary>Tasks</summary><br>`bigbench:play_dialog_same_or_different.mul.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:play_dialog_same_or_different.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:play_dialog_same_or_different.mul.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:play_dialog_same_or_different.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:play_dialog_same_or_different.mul.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:play_dialog_same_or_different.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:play_dialog_same_or_different.mul.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:play_dialog_same_or_different.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Determine if nearby lines in a Shakespeare play were spoken by the same individual</details> | Multiple-Choice
[polish_sequence_labeling](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/polish_sequence_labeling/) | <details><summary>Tasks</summary><br>`bigbench:polish_sequence_labeling.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:polish_sequence_labeling.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:polish_sequence_labeling.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:polish_sequence_labeling.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:polish_sequence_labeling.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:polish_sequence_labeling.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:polish_sequence_labeling.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:polish_sequence_labeling.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Perform named-entity recognition, temporal-expression extraction and event extraction on Polish texts</details> | Generative, Scoring
[presuppositions_as_nli](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/presuppositions_as_nli/) | <details><summary>Tasks</summary><br>`bigbench:presuppositions_as_nli.mul.t5_default_vocab.0_shot.735_examples`<br>`bigbench:presuppositions_as_nli.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:presuppositions_as_nli.mul.t5_default_vocab.1_shot.735_examples`<br>`bigbench:presuppositions_as_nli.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:presuppositions_as_nli.mul.t5_default_vocab.2_shot.735_examples`<br>`bigbench:presuppositions_as_nli.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:presuppositions_as_nli.mul.t5_default_vocab.3_shot.735_examples`<br>`bigbench:presuppositions_as_nli.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Determine whether the first sentence entails or contradicts the second</details> | Multiple-Choice
[qa_wikidata](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/qa_wikidata/) | <details><summary>Tasks</summary><br>`bigbench:qa_wikidata.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:qa_wikidata.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:qa_wikidata.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:qa_wikidata.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:qa_wikidata.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:qa_wikidata.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:qa_wikidata.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:qa_wikidata.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer simple prompts for questions formed from randomly-sampled Wikidata fact triples</details> | Generative, Scoring
[question_selection](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/question_selection/) | <details><summary>Tasks</summary><br>`bigbench:question_selection.mul.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:question_selection.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:question_selection.mul.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:question_selection.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:question_selection.mul.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:question_selection.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:question_selection.mul.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:question_selection.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Given a short answer along with its context, select the most appropriate question which has the given short answer as its answer</details> | Multiple-Choice
[reasoning_about_colored_objects](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/reasoning_about_colored_objects/) | <details><summary>Tasks</summary><br>`bigbench:reasoning_about_colored_objects.mul.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:reasoning_about_colored_objects.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:reasoning_about_colored_objects.mul.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:reasoning_about_colored_objects.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:reasoning_about_colored_objects.mul.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:reasoning_about_colored_objects.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:reasoning_about_colored_objects.mul.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:reasoning_about_colored_objects.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer extremely simple questions about the colors of objects on a surface</details> | Multiple-Choice
[repeat_copy_logic](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/repeat_copy_logic/) | <details><summary>Tasks</summary><br>`bigbench:repeat_copy_logic.mix.t5_default_vocab.0_shot.32_examples`<br>`bigbench:repeat_copy_logic.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:repeat_copy_logic.mix.t5_default_vocab.1_shot.32_examples`<br>`bigbench:repeat_copy_logic.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:repeat_copy_logic.mix.t5_default_vocab.2_shot.32_examples`<br>`bigbench:repeat_copy_logic.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:repeat_copy_logic.mix.t5_default_vocab.3_shot.32_examples`<br>`bigbench:repeat_copy_logic.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Follow basic instructions to repeat parts of the input</details> | Generative, Scoring
[rephrase](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/rephrase/) | <details><summary>Tasks</summary><br>`bigbench:rephrase.mix.t5_default_vocab.0_shot.78_examples`<br>`bigbench:rephrase.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:rephrase.mix.t5_default_vocab.1_shot.78_examples`<br>`bigbench:rephrase.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:rephrase.mix.t5_default_vocab.2_shot.78_examples`<br>`bigbench:rephrase.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:rephrase.mix.t5_default_vocab.3_shot.78_examples`<br>`bigbench:rephrase.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Rephrase the given sentence so that it retains its meaning, but contains the given keyword</details> | Generative, Scoring
[riddle_sense](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/riddle_sense/) | <details><summary>Tasks</summary><br>`bigbench:riddle_sense.mul.t5_default_vocab.0_shot.49_examples`<br>`bigbench:riddle_sense.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:riddle_sense.mul.t5_default_vocab.1_shot.49_examples`<br>`bigbench:riddle_sense.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:riddle_sense.mul.t5_default_vocab.2_shot.49_examples`<br>`bigbench:riddle_sense.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:riddle_sense.mul.t5_default_vocab.3_shot.49_examples`<br>`bigbench:riddle_sense.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer riddle-style commonsense questions (multiple-choice)</details> | Multiple-Choice
[ruin_names](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/ruin_names/) | <details><summary>Tasks</summary><br>`bigbench:ruin_names.mul.t5_default_vocab.0_shot.448_examples`<br>`bigbench:ruin_names.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:ruin_names.mul.t5_default_vocab.1_shot.448_examples`<br>`bigbench:ruin_names.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:ruin_names.mul.t5_default_vocab.2_shot.448_examples`<br>`bigbench:ruin_names.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:ruin_names.mul.t5_default_vocab.3_shot.448_examples`<br>`bigbench:ruin_names.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Select the humorous edit that 'ruins' the input movie or musical artist name</details> | Multiple-Choice
[salient_translation_error_detection](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/salient_translation_error_detection/) | <details><summary>Tasks</summary><br>`bigbench:salient_translation_error_detection.mul.t5_default_vocab.0_shot.998_examples`<br>`bigbench:salient_translation_error_detection.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:salient_translation_error_detection.mul.t5_default_vocab.1_shot.998_examples`<br>`bigbench:salient_translation_error_detection.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:salient_translation_error_detection.mul.t5_default_vocab.2_shot.998_examples`<br>`bigbench:salient_translation_error_detection.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:salient_translation_error_detection.mul.t5_default_vocab.3_shot.998_examples`<br>`bigbench:salient_translation_error_detection.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Detect the type of error in an English translation of a German source sentence</details> | Multiple-Choice
[scientific_press_release](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/scientific_press_release/) | <details><summary>Tasks</summary><br>`bigbench:scientific_press_release.mix.t5_default_vocab.0_shot.50_examples`<br>`bigbench:scientific_press_release.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:scientific_press_release.mix.t5_default_vocab.1_shot.50_examples`<br>`bigbench:scientific_press_release.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:scientific_press_release.mix.t5_default_vocab.2_shot.50_examples`<br>`bigbench:scientific_press_release.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:scientific_press_release.mix.t5_default_vocab.3_shot.50_examples`<br>`bigbench:scientific_press_release.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Generate readable press-release titles of scientific headlines</details> | Generative, Scoring
[semantic_parsing_in_context_sparc](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/semantic_parsing_in_context_sparc/) | <details><summary>Tasks</summary><br>`bigbench:semantic_parsing_in_context_sparc.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:semantic_parsing_in_context_sparc.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:semantic_parsing_in_context_sparc.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:semantic_parsing_in_context_sparc.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:semantic_parsing_in_context_sparc.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:semantic_parsing_in_context_sparc.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:semantic_parsing_in_context_sparc.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:semantic_parsing_in_context_sparc.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Map natural-language utterances into SQL queries that can be executed on databases</details> | Generative, Scoring
[semantic_parsing_spider](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/semantic_parsing_spider/) | <details><summary>Tasks</summary><br>`bigbench:semantic_parsing_spider.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:semantic_parsing_spider.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:semantic_parsing_spider.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:semantic_parsing_spider.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:semantic_parsing_spider.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:semantic_parsing_spider.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:semantic_parsing_spider.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:semantic_parsing_spider.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Map a natural-language utterance into an SQL query that can be executed on databases</details> | Generative, Scoring
[sentence_ambiguity](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/sentence_ambiguity/) | <details><summary>Tasks</summary><br>`bigbench:sentence_ambiguity.mul.t5_default_vocab.0_shot.60_examples`<br>`bigbench:sentence_ambiguity.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:sentence_ambiguity.mul.t5_default_vocab.1_shot.60_examples`<br>`bigbench:sentence_ambiguity.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:sentence_ambiguity.mul.t5_default_vocab.2_shot.60_examples`<br>`bigbench:sentence_ambiguity.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:sentence_ambiguity.mul.t5_default_vocab.3_shot.60_examples`<br>`bigbench:sentence_ambiguity.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify the truth or falsehood of potentially misleading sentences</details> | Multiple-Choice
[similarities_abstraction](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/similarities_abstraction/) | <details><summary>Tasks</summary><br>`bigbench:similarities_abstraction.mix.t5_default_vocab.0_shot.76_examples`<br>`bigbench:similarities_abstraction.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:similarities_abstraction.mix.t5_default_vocab.1_shot.76_examples`<br>`bigbench:similarities_abstraction.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:similarities_abstraction.mix.t5_default_vocab.2_shot.76_examples`<br>`bigbench:similarities_abstraction.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:similarities_abstraction.mix.t5_default_vocab.3_shot.76_examples`<br>`bigbench:similarities_abstraction.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify how two objects are similar</details> | Generative, Multiple-Choice, Scoring
[simple_arithmetic_json](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/simple_arithmetic_json/) | <details><summary>Tasks</summary><br>`bigbench:simple_arithmetic_json.mix.t5_default_vocab.0_shot.30_examples`<br>`bigbench:simple_arithmetic_json.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:simple_arithmetic_json.mix.t5_default_vocab.1_shot.30_examples`<br>`bigbench:simple_arithmetic_json.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:simple_arithmetic_json.mix.t5_default_vocab.2_shot.30_examples`<br>`bigbench:simple_arithmetic_json.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:simple_arithmetic_json.mix.t5_default_vocab.3_shot.30_examples`<br>`bigbench:simple_arithmetic_json.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>A template task to be used as an example during task development (simple variable-difficulty arithmetic)</details> | Generative, Scoring
[simple_arithmetic_json_multiple_choice](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/simple_arithmetic_json_multiple_choice/) | <details><summary>Tasks</summary><br>`bigbench:simple_arithmetic_json_multiple_choice.mul.t5_default_vocab.0_shot.8_examples`<br>`bigbench:simple_arithmetic_json_multiple_choice.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:simple_arithmetic_json_multiple_choice.mul.t5_default_vocab.1_shot.8_examples`<br>`bigbench:simple_arithmetic_json_multiple_choice.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:simple_arithmetic_json_multiple_choice.mul.t5_default_vocab.2_shot.8_examples`<br>`bigbench:simple_arithmetic_json_multiple_choice.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:simple_arithmetic_json_multiple_choice.mul.t5_default_vocab.3_shot.8_examples`<br>`bigbench:simple_arithmetic_json_multiple_choice.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>A template task to be used as an example during task development (simple variable-difficulty arithmetic)</details> | Multiple-Choice
[simple_arithmetic_multiple_targets_json](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/simple_arithmetic_multiple_targets_json/) | <details><summary>Tasks</summary><br>`bigbench:simple_arithmetic_multiple_targets_json.mix.t5_default_vocab.0_shot.10_examples`<br>`bigbench:simple_arithmetic_multiple_targets_json.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:simple_arithmetic_multiple_targets_json.mix.t5_default_vocab.1_shot.10_examples`<br>`bigbench:simple_arithmetic_multiple_targets_json.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:simple_arithmetic_multiple_targets_json.mix.t5_default_vocab.2_shot.10_examples`<br>`bigbench:simple_arithmetic_multiple_targets_json.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:simple_arithmetic_multiple_targets_json.mix.t5_default_vocab.3_shot.10_examples`<br>`bigbench:simple_arithmetic_multiple_targets_json.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>A template task to be used as an example during task development (simple variable-difficulty arithmetic)</details> | Generative, Scoring
[simple_ethical_questions](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/simple_ethical_questions/) | <details><summary>Tasks</summary><br>`bigbench:simple_ethical_questions.mul.t5_default_vocab.0_shot.115_examples`<br>`bigbench:simple_ethical_questions.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:simple_ethical_questions.mul.t5_default_vocab.1_shot.115_examples`<br>`bigbench:simple_ethical_questions.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:simple_ethical_questions.mul.t5_default_vocab.2_shot.115_examples`<br>`bigbench:simple_ethical_questions.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:simple_ethical_questions.mul.t5_default_vocab.3_shot.115_examples`<br>`bigbench:simple_ethical_questions.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Evaluates if a model is capable of identifying which ethical choice best aligns with human judgement</details> | Multiple-Choice
[simple_text_editing](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/simple_text_editing/) | <details><summary>Tasks</summary><br>`bigbench:simple_text_editing.mix.t5_default_vocab.0_shot.47_examples`<br>`bigbench:simple_text_editing.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:simple_text_editing.mix.t5_default_vocab.1_shot.47_examples`<br>`bigbench:simple_text_editing.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:simple_text_editing.mix.t5_default_vocab.2_shot.47_examples`<br>`bigbench:simple_text_editing.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:simple_text_editing.mix.t5_default_vocab.3_shot.47_examples`<br>`bigbench:simple_text_editing.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Carry out basic text-editing operations</details> | Generative, Scoring
[snarks](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/snarks/) | <details><summary>Tasks</summary><br>`bigbench:snarks.mul.t5_default_vocab.0_shot.181_examples`<br>`bigbench:snarks.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:snarks.mul.t5_default_vocab.1_shot.181_examples`<br>`bigbench:snarks.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:snarks.mul.t5_default_vocab.2_shot.181_examples`<br>`bigbench:snarks.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:snarks.mul.t5_default_vocab.3_shot.181_examples`<br>`bigbench:snarks.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Determine which of two sentences is sarcastic</details> | Multiple-Choice
[social_iqa](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/social_iqa/) | <details><summary>Tasks</summary><br>`bigbench:social_iqa.mul.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:social_iqa.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:social_iqa.mul.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:social_iqa.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:social_iqa.mul.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:social_iqa.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:social_iqa.mul.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:social_iqa.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer questions about the motivations, emotional reactions, and preceding and following events surrounding interpersonal situations</details> | Multiple-Choice
[social_support](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/social_support/) | <details><summary>Tasks</summary><br>`bigbench:social_support.mul.t5_default_vocab.0_shot.897_examples`<br>`bigbench:social_support.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:social_support.mul.t5_default_vocab.1_shot.897_examples`<br>`bigbench:social_support.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:social_support.mul.t5_default_vocab.2_shot.897_examples`<br>`bigbench:social_support.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:social_support.mul.t5_default_vocab.3_shot.897_examples`<br>`bigbench:social_support.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>The task aims to measure the ability of pre-trained models on understanding supportive (and unsupportive) language uses</details> | Multiple-Choice
[sports_understanding](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/sports_understanding/) | <details><summary>Tasks</summary><br>`bigbench:sports_understanding.mul.t5_default_vocab.0_shot.986_examples`<br>`bigbench:sports_understanding.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:sports_understanding.mul.t5_default_vocab.1_shot.986_examples`<br>`bigbench:sports_understanding.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:sports_understanding.mul.t5_default_vocab.2_shot.986_examples`<br>`bigbench:sports_understanding.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:sports_understanding.mul.t5_default_vocab.3_shot.986_examples`<br>`bigbench:sports_understanding.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Determine whether an artificially constructed sentence relating to sports is plausible or implausible</details> | Multiple-Choice
[strategyqa](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/strategyqa/) | <details><summary>Tasks</summary><br>`bigbench:strategyqa.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:strategyqa.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:strategyqa.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:strategyqa.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:strategyqa.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:strategyqa.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:strategyqa.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:strategyqa.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer questions in which the required reasoning steps are implicit in the question</details> | Generative, Multiple-Choice, Scoring
[sufficient_information](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/sufficient_information/) | <details><summary>Tasks</summary><br>`bigbench:sufficient_information.mix.t5_default_vocab.0_shot.39_examples`<br>`bigbench:sufficient_information.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:sufficient_information.mix.t5_default_vocab.1_shot.39_examples`<br>`bigbench:sufficient_information.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:sufficient_information.mix.t5_default_vocab.2_shot.39_examples`<br>`bigbench:sufficient_information.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:sufficient_information.mix.t5_default_vocab.3_shot.39_examples`<br>`bigbench:sufficient_information.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer questions based on provided context, or state that you do not know the answer</details> | Generative, Scoring
[suicide_risk](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/suicide_risk/) | <details><summary>Tasks</summary><br>`bigbench:suicide_risk.mul.t5_default_vocab.0_shot.40_examples`<br>`bigbench:suicide_risk.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:suicide_risk.mul.t5_default_vocab.1_shot.40_examples`<br>`bigbench:suicide_risk.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:suicide_risk.mul.t5_default_vocab.2_shot.40_examples`<br>`bigbench:suicide_risk.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:suicide_risk.mul.t5_default_vocab.3_shot.40_examples`<br>`bigbench:suicide_risk.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Evaluates whether large language models are capable of matching human expert assessment of suicide risk from written texts</details> | Multiple-Choice
[swahili_english_proverbs](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/swahili_english_proverbs/) | <details><summary>Tasks</summary><br>`bigbench:swahili_english_proverbs.mul.t5_default_vocab.0_shot.153_examples`<br>`bigbench:swahili_english_proverbs.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:swahili_english_proverbs.mul.t5_default_vocab.1_shot.153_examples`<br>`bigbench:swahili_english_proverbs.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:swahili_english_proverbs.mul.t5_default_vocab.2_shot.153_examples`<br>`bigbench:swahili_english_proverbs.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:swahili_english_proverbs.mul.t5_default_vocab.3_shot.153_examples`<br>`bigbench:swahili_english_proverbs.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>For a given proverb in Kiswahili, choose a proverb in English which is closest in meaning</details> | Multiple-Choice
[swedish_to_german_proverbs](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/swedish_to_german_proverbs/) | <details><summary>Tasks</summary><br>`bigbench:swedish_to_german_proverbs.mul.t5_default_vocab.0_shot.72_examples`<br>`bigbench:swedish_to_german_proverbs.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:swedish_to_german_proverbs.mul.t5_default_vocab.1_shot.72_examples`<br>`bigbench:swedish_to_german_proverbs.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:swedish_to_german_proverbs.mul.t5_default_vocab.2_shot.72_examples`<br>`bigbench:swedish_to_german_proverbs.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:swedish_to_german_proverbs.mul.t5_default_vocab.3_shot.72_examples`<br>`bigbench:swedish_to_german_proverbs.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Find a German proverb with the same meaning as the provided Swedish proverb</details> | Multiple-Choice
[temporal_sequences](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/temporal_sequences/) | <details><summary>Tasks</summary><br>`bigbench:temporal_sequences.mul.t5_default_vocab.0_shot.1000_examples`<br>`bigbench:temporal_sequences.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:temporal_sequences.mul.t5_default_vocab.1_shot.1000_examples`<br>`bigbench:temporal_sequences.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:temporal_sequences.mul.t5_default_vocab.2_shot.1000_examples`<br>`bigbench:temporal_sequences.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:temporal_sequences.mul.t5_default_vocab.3_shot.1000_examples`<br>`bigbench:temporal_sequences.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Answer questions about which times certain events could have occurred</details> | Multiple-Choice
[tense](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/tense/) | <details><summary>Tasks</summary><br>`bigbench:tense.mix.t5_default_vocab.0_shot.286_examples`<br>`bigbench:tense.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:tense.mix.t5_default_vocab.1_shot.286_examples`<br>`bigbench:tense.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:tense.mix.t5_default_vocab.2_shot.286_examples`<br>`bigbench:tense.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:tense.mix.t5_default_vocab.3_shot.286_examples`<br>`bigbench:tense.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Modify the tense of a given sentence</details> | Generative, Scoring
[timedial](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/timedial/) | <details><summary>Tasks</summary><br>`bigbench:timedial.mul.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:timedial.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:timedial.mul.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:timedial.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:timedial.mul.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:timedial.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:timedial.mul.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:timedial.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Pick the correct choice for a masked (temporal) span given the dialog context</details> | Multiple-Choice
[topical_chat](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/topical_chat/) | <details><summary>Tasks</summary><br>`bigbench:topical_chat.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:topical_chat.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:topical_chat.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:topical_chat.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:topical_chat.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:topical_chat.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:topical_chat.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:topical_chat.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Open-domain response generation</details> | Generative, Scoring
[understanding_fables](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/understanding_fables/) | <details><summary>Tasks</summary><br>`bigbench:understanding_fables.mul.t5_default_vocab.0_shot.189_examples`<br>`bigbench:understanding_fables.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:understanding_fables.mul.t5_default_vocab.1_shot.189_examples`<br>`bigbench:understanding_fables.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:understanding_fables.mul.t5_default_vocab.2_shot.189_examples`<br>`bigbench:understanding_fables.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:understanding_fables.mul.t5_default_vocab.3_shot.189_examples`<br>`bigbench:understanding_fables.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify the most suitable moral for a given fable</details> | Multiple-Choice
[vitaminc_fact_verification](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/vitaminc_fact_verification/) | <details><summary>Tasks</summary><br>`bigbench:vitaminc_fact_verification.mul.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:vitaminc_fact_verification.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:vitaminc_fact_verification.mul.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:vitaminc_fact_verification.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:vitaminc_fact_verification.mul.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:vitaminc_fact_verification.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:vitaminc_fact_verification.mul.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:vitaminc_fact_verification.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Identify whether a claim is True or False based on the given context</details> | Multiple-Choice
[what_is_the_tao](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/what_is_the_tao/) | <details><summary>Tasks</summary><br>`bigbench:what_is_the_tao.mul.t5_default_vocab.0_shot.36_examples`<br>`bigbench:what_is_the_tao.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:what_is_the_tao.mul.t5_default_vocab.1_shot.36_examples`<br>`bigbench:what_is_the_tao.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:what_is_the_tao.mul.t5_default_vocab.2_shot.36_examples`<br>`bigbench:what_is_the_tao.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:what_is_the_tao.mul.t5_default_vocab.3_shot.36_examples`<br>`bigbench:what_is_the_tao.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Compare the stylistic elements of different translations of a complex philosophical text</details> | Multiple-Choice
[which_wiki_edit](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/which_wiki_edit/) | <details><summary>Tasks</summary><br>`bigbench:which_wiki_edit.mul.t5_default_vocab.0_shot.571_examples`<br>`bigbench:which_wiki_edit.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:which_wiki_edit.mul.t5_default_vocab.1_shot.571_examples`<br>`bigbench:which_wiki_edit.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:which_wiki_edit.mul.t5_default_vocab.2_shot.571_examples`<br>`bigbench:which_wiki_edit.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:which_wiki_edit.mul.t5_default_vocab.3_shot.571_examples`<br>`bigbench:which_wiki_edit.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Match a recent Wikipedia revision to its corresponding edit message</details> | Multiple-Choice
[winowhy](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/winowhy/) | <details><summary>Tasks</summary><br>`bigbench:winowhy.mul.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:winowhy.mul.t5_default_vocab.0_shot.all_examples`<br>`bigbench:winowhy.mul.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:winowhy.mul.t5_default_vocab.1_shot.all_examples`<br>`bigbench:winowhy.mul.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:winowhy.mul.t5_default_vocab.2_shot.all_examples`<br>`bigbench:winowhy.mul.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:winowhy.mul.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Evaluate the reasoning in answering Winograd Schema Challenge questions</details> | Multiple-Choice
[word_sorting](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/word_sorting/) | <details><summary>Tasks</summary><br>`bigbench:word_sorting.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:word_sorting.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:word_sorting.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:word_sorting.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:word_sorting.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:word_sorting.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:word_sorting.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:word_sorting.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Sort a list of words</details> | Generative, Scoring
[word_unscrambling](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/word_unscrambling/) | <details><summary>Tasks</summary><br>`bigbench:word_unscrambling.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:word_unscrambling.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:word_unscrambling.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:word_unscrambling.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:word_unscrambling.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:word_unscrambling.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:word_unscrambling.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:word_unscrambling.mix.t5_default_vocab.3_shot.all_examples`</details>  | - | <details><summary>Description</summary><br>Unscramble the given letters to form an English word</details> | Generative, Scoring
[abstract_narrative_understanding](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/abstract_narrative_understanding/) | <details><summary>Tasks</summary><br>`bigbench:abstract_narrative_understanding.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:abstract_narrative_understanding.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:abstract_narrative_understanding.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:abstract_narrative_understanding.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:abstract_narrative_understanding.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:abstract_narrative_understanding.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:abstract_narrative_understanding.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:abstract_narrative_understanding.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>4_distractors<br>99_distractors<br>9_distractors</details> | <details><summary>Description</summary><br>Given a narrative, choose the most related proverb</details> | Multiple-Choice
[arithmetic](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/arithmetic/) | <details><summary>Tasks</summary><br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.23_examples.1_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.1_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.1_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.1_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.2_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.2_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.2_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.2_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.3_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.3_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.3_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.3_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.4_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.4_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.4_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.4_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.5_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.5_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.5_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.51_examples.5_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.1_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.1_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.1_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.1_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.2_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.2_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.2_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.2_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.3_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.3_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.3_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.3_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.4_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.4_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.4_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.4_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.5_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.5_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.5_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples.5_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.23_examples.1_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.1_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.1_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.1_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.2_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.2_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.2_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.2_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.3_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.3_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.3_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.3_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.4_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.4_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.4_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.4_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.5_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.5_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.5_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.51_examples.5_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.1_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.1_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.1_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.1_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.2_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.2_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.2_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.2_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.3_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.3_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.3_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.3_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.4_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.4_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.4_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.4_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.5_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.5_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.5_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples.5_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.23_examples.1_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.1_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.1_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.1_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.2_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.2_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.2_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.2_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.3_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.3_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.3_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.3_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.4_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.4_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.4_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.4_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.5_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.5_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.5_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.51_examples.5_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.1_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.1_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.1_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.1_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.2_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.2_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.2_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.2_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.3_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.3_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.3_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.3_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.4_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.4_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.4_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.4_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.5_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.5_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.5_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples.5_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.23_examples.1_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.1_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.1_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.1_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.2_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.2_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.2_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.2_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.3_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.3_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.3_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.3_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.4_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.4_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.4_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.4_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.5_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.5_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.5_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.51_examples.5_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.1_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.1_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.1_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.1_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.2_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.2_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.2_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.2_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.3_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.3_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.3_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.3_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.4_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.4_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.4_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.4_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.5_digit_addition`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.5_digit_division`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.5_digit_multiplication`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples.5_digit_subtraction`<br>`bigbench:arithmetic.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>1_digit_addition<br>1_digit_division<br>1_digit_multiplication<br>1_digit_subtraction<br>2_digit_addition<br>2_digit_division<br>2_digit_multiplication<br>2_digit_subtraction<br>3_digit_addition<br>3_digit_division<br>3_digit_multiplication<br>3_digit_subtraction<br>4_digit_addition<br>4_digit_division<br>4_digit_multiplication<br>4_digit_subtraction<br>5_digit_addition<br>5_digit_division<br>5_digit_multiplication<br>5_digit_subtraction</details> | <details><summary>Description</summary><br>Perform the four basic arithmetic operations</details> | Generative, Multiple-Choice, Scoring
[authorship_verification](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/authorship_verification/) | <details><summary>Tasks</summary><br>`bigbench:authorship_verification.mix.t5_default_vocab.0_shot.880_examples`<br>`bigbench:authorship_verification.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:authorship_verification.mix.t5_default_vocab.1_shot.880_examples`<br>`bigbench:authorship_verification.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:authorship_verification.mix.t5_default_vocab.2_shot.880_examples`<br>`bigbench:authorship_verification.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:authorship_verification.mix.t5_default_vocab.3_shot.880_examples`<br>`bigbench:authorship_verification.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>non_swapped<br>swapped</details> | <details><summary>Description</summary><br>Identify which of the text passages given as choices was written by the same author as the text passage given as the reference</details> | Multiple-Choice
[bbq_lite_json](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/bbq_lite_json/) | <details><summary>Tasks</summary><br>`bigbench:bbq_lite_json.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:bbq_lite_json.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:bbq_lite_json.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:bbq_lite_json.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:bbq_lite_json.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:bbq_lite_json.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:bbq_lite_json.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:bbq_lite_json.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>bbq_lite_json_age_ambig<br>bbq_lite_json_age_disambig<br>bbq_lite_json_disability_status_ambig<br>bbq_lite_json_disability_status_disambig<br>bbq_lite_json_gender_identity_ambig<br>bbq_lite_json_gender_identity_disambig<br>bbq_lite_json_nationality_ambig<br>bbq_lite_json_nationality_disambig<br>bbq_lite_json_physical_appearance_ambig<br>bbq_lite_json_physical_appearance_disambig<br>bbq_lite_json_race_ethnicity_ambig<br>bbq_lite_json_race_ethnicity_disambig<br>bbq_lite_json_religion_ambig<br>bbq_lite_json_religion_disambig<br>bbq_lite_json_ses_ambig<br>bbq_lite_json_ses_disambig<br>bbq_lite_json_sexual_orientation_ambig<br>bbq_lite_json_sexual_orientation_disambig</details> | <details><summary>Description</summary><br>A social bias measurement task for multiple choice question answering models</details> | Multiple-Choice
[cause_and_effect](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/cause_and_effect/) | <details><summary>Tasks</summary><br>`bigbench:cause_and_effect.mix.t5_default_vocab.0_shot.153_examples`<br>`bigbench:cause_and_effect.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:cause_and_effect.mix.t5_default_vocab.1_shot.153_examples`<br>`bigbench:cause_and_effect.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:cause_and_effect.mix.t5_default_vocab.2_shot.153_examples`<br>`bigbench:cause_and_effect.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:cause_and_effect.mix.t5_default_vocab.3_shot.153_examples`<br>`bigbench:cause_and_effect.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>one_sentence<br>one_sentence_no_prompt<br>two_sentences</details> | <details><summary>Description</summary><br>Answer multiple-choice questions distinguishing cause and effect</details> | Multiple-Choice
[chess_state_tracking](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/chess_state_tracking/) | <details><summary>Tasks</summary><br>`bigbench:chess_state_tracking.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.0_shot.170_examples.real_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.0_shot.170_examples.real_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.0_shot.170_examples.real_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.0_shot.170_examples.synthetic_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.0_shot.170_examples.synthetic_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.0_shot.170_examples.synthetic_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.0_shot.all_examples.real_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.0_shot.all_examples.real_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.0_shot.all_examples.real_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.0_shot.all_examples.synthetic_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.0_shot.all_examples.synthetic_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.0_shot.all_examples.synthetic_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.1_shot.170_examples.real_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.1_shot.170_examples.real_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.1_shot.170_examples.real_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.1_shot.170_examples.synthetic_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.1_shot.170_examples.synthetic_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.1_shot.170_examples.synthetic_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.1_shot.all_examples.real_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.1_shot.all_examples.real_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.1_shot.all_examples.real_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.1_shot.all_examples.synthetic_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.1_shot.all_examples.synthetic_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.1_shot.all_examples.synthetic_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.2_shot.170_examples.real_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.2_shot.170_examples.real_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.2_shot.170_examples.real_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.2_shot.170_examples.synthetic_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.2_shot.170_examples.synthetic_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.2_shot.170_examples.synthetic_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.2_shot.all_examples.real_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.2_shot.all_examples.real_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.2_shot.all_examples.real_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.2_shot.all_examples.synthetic_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.2_shot.all_examples.synthetic_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.2_shot.all_examples.synthetic_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.3_shot.170_examples.real_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.3_shot.170_examples.real_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.3_shot.170_examples.real_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.3_shot.170_examples.synthetic_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.3_shot.170_examples.synthetic_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.3_shot.170_examples.synthetic_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.3_shot.all_examples.real_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.3_shot.all_examples.real_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.3_shot.all_examples.real_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.3_shot.all_examples.synthetic_long`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.3_shot.all_examples.synthetic_medium`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.3_shot.all_examples.synthetic_short`<br>`bigbench:chess_state_tracking.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>real_long<br>real_medium<br>real_short<br>synthetic_long<br>synthetic_medium<br>synthetic_short</details> | <details><summary>Description</summary><br>Identify legal moves in the given chess position</details> | Generative, Scoring
[cifar10_classification](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/cifar10_classification/) | <details><summary>Tasks</summary><br>`bigbench:cifar10_classification.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:cifar10_classification.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:cifar10_classification.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:cifar10_classification.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:cifar10_classification.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:cifar10_classification.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:cifar10_classification.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:cifar10_classification.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>base64<br>hex</details> | <details><summary>Description</summary><br>Classify CIFAR10 images encoded in various ways</details> | Multiple-Choice
[color](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/color/) | <details><summary>Tasks</summary><br>`bigbench:color.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:color.mix.t5_default_vocab.0_shot.256_examples.hcl`<br>`bigbench:color.mix.t5_default_vocab.0_shot.256_examples.hex`<br>`bigbench:color.mix.t5_default_vocab.0_shot.256_examples.hsl`<br>`bigbench:color.mix.t5_default_vocab.0_shot.256_examples.rgb`<br>`bigbench:color.mix.t5_default_vocab.0_shot.all_examples.hcl`<br>`bigbench:color.mix.t5_default_vocab.0_shot.all_examples.hex`<br>`bigbench:color.mix.t5_default_vocab.0_shot.all_examples.hsl`<br>`bigbench:color.mix.t5_default_vocab.0_shot.all_examples.rgb`<br>`bigbench:color.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:color.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:color.mix.t5_default_vocab.1_shot.256_examples.hcl`<br>`bigbench:color.mix.t5_default_vocab.1_shot.256_examples.hex`<br>`bigbench:color.mix.t5_default_vocab.1_shot.256_examples.hsl`<br>`bigbench:color.mix.t5_default_vocab.1_shot.256_examples.rgb`<br>`bigbench:color.mix.t5_default_vocab.1_shot.all_examples.hcl`<br>`bigbench:color.mix.t5_default_vocab.1_shot.all_examples.hex`<br>`bigbench:color.mix.t5_default_vocab.1_shot.all_examples.hsl`<br>`bigbench:color.mix.t5_default_vocab.1_shot.all_examples.rgb`<br>`bigbench:color.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:color.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:color.mix.t5_default_vocab.2_shot.256_examples.hcl`<br>`bigbench:color.mix.t5_default_vocab.2_shot.256_examples.hex`<br>`bigbench:color.mix.t5_default_vocab.2_shot.256_examples.hsl`<br>`bigbench:color.mix.t5_default_vocab.2_shot.256_examples.rgb`<br>`bigbench:color.mix.t5_default_vocab.2_shot.all_examples.hcl`<br>`bigbench:color.mix.t5_default_vocab.2_shot.all_examples.hex`<br>`bigbench:color.mix.t5_default_vocab.2_shot.all_examples.hsl`<br>`bigbench:color.mix.t5_default_vocab.2_shot.all_examples.rgb`<br>`bigbench:color.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:color.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:color.mix.t5_default_vocab.3_shot.256_examples.hcl`<br>`bigbench:color.mix.t5_default_vocab.3_shot.256_examples.hex`<br>`bigbench:color.mix.t5_default_vocab.3_shot.256_examples.hsl`<br>`bigbench:color.mix.t5_default_vocab.3_shot.256_examples.rgb`<br>`bigbench:color.mix.t5_default_vocab.3_shot.all_examples.hcl`<br>`bigbench:color.mix.t5_default_vocab.3_shot.all_examples.hex`<br>`bigbench:color.mix.t5_default_vocab.3_shot.all_examples.hsl`<br>`bigbench:color.mix.t5_default_vocab.3_shot.all_examples.rgb`<br>`bigbench:color.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>hcl<br>hex<br>hsl<br>rgb</details> | <details><summary>Description</summary><br>Identify the color specified by the given RGB, HEX, HSL, or HCL encoding</details> | Generative, Multiple-Choice, Scoring
[conceptual_combinations](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/conceptual_combinations/) | <details><summary>Tasks</summary><br>`bigbench:conceptual_combinations.mix.t5_default_vocab.0_shot.103_examples`<br>`bigbench:conceptual_combinations.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:conceptual_combinations.mix.t5_default_vocab.1_shot.103_examples`<br>`bigbench:conceptual_combinations.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:conceptual_combinations.mix.t5_default_vocab.2_shot.103_examples`<br>`bigbench:conceptual_combinations.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:conceptual_combinations.mix.t5_default_vocab.3_shot.103_examples`<br>`bigbench:conceptual_combinations.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>contradictions<br>emergent_properties<br>fanciful_fictional_combinations<br>homonyms<br>invented_words<br>surprising_uncommon_combinations</details> | <details><summary>Description</summary><br>Understand conceptual combinations in appropriate contexts</details> | Multiple-Choice
[conlang_translation](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/conlang_translation/) | <details><summary>Tasks</summary><br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.10_examples.adna_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.10_examples.adna_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.10_examples.atikamp__from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.10_examples.atikamp__to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.10_examples.gornam_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.10_examples.holuan_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.10_examples.holuan_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.10_examples.mkafala_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.10_examples.mkafala_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.10_examples.pp_english_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.10_examples.pp_english_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.10_examples.unapuri_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.10_examples.vaomi_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.10_examples.vaomi_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.11_examples.unapuri_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.13_examples.gornam_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.164_examples`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.adna_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.adna_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.atikamp__from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.atikamp__to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.gornam_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.gornam_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.holuan_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.holuan_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.mkafala_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.mkafala_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.pp_english_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.pp_english_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.unapuri_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.unapuri_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.vaomi_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples.vaomi_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.10_examples.adna_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.10_examples.adna_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.10_examples.atikamp__from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.10_examples.atikamp__to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.10_examples.gornam_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.10_examples.holuan_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.10_examples.holuan_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.10_examples.mkafala_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.10_examples.mkafala_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.10_examples.pp_english_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.10_examples.pp_english_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.10_examples.unapuri_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.10_examples.vaomi_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.10_examples.vaomi_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.11_examples.unapuri_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.13_examples.gornam_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.164_examples`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.adna_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.adna_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.atikamp__from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.atikamp__to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.gornam_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.gornam_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.holuan_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.holuan_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.mkafala_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.mkafala_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.pp_english_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.pp_english_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.unapuri_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.unapuri_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.vaomi_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples.vaomi_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.10_examples.adna_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.10_examples.adna_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.10_examples.atikamp__from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.10_examples.atikamp__to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.10_examples.gornam_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.10_examples.holuan_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.10_examples.holuan_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.10_examples.mkafala_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.10_examples.mkafala_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.10_examples.pp_english_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.10_examples.pp_english_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.10_examples.unapuri_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.10_examples.vaomi_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.10_examples.vaomi_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.11_examples.unapuri_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.13_examples.gornam_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.164_examples`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.adna_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.adna_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.atikamp__from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.atikamp__to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.gornam_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.gornam_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.holuan_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.holuan_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.mkafala_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.mkafala_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.pp_english_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.pp_english_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.unapuri_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.unapuri_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.vaomi_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples.vaomi_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.10_examples.adna_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.10_examples.adna_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.10_examples.atikamp__from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.10_examples.atikamp__to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.10_examples.gornam_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.10_examples.holuan_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.10_examples.holuan_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.10_examples.mkafala_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.10_examples.mkafala_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.10_examples.pp_english_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.10_examples.pp_english_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.10_examples.unapuri_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.10_examples.vaomi_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.10_examples.vaomi_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.11_examples.unapuri_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.13_examples.gornam_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.164_examples`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.adna_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.adna_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.atikamp__from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.atikamp__to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.gornam_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.gornam_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.holuan_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.holuan_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.mkafala_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.mkafala_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.pp_english_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.pp_english_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.unapuri_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.unapuri_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.vaomi_from`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples.vaomi_to`<br>`bigbench:conlang_translation.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>adna_from<br>adna_to<br>atikamp__from<br>atikamp__to<br>gornam_from<br>gornam_to<br>holuan_from<br>holuan_to<br>mkafala_from<br>mkafala_to<br>pp_english_from<br>pp_english_to<br>unapuri_from<br>unapuri_to<br>vaomi_from<br>vaomi_to</details> | <details><summary>Description</summary><br>Decipher language rules and lexicon from a few examples</details> | Generative, Scoring
[cs_algorithms](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/cs_algorithms/) | <details><summary>Tasks</summary><br>`bigbench:cs_algorithms.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:cs_algorithms.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:cs_algorithms.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:cs_algorithms.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:cs_algorithms.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:cs_algorithms.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:cs_algorithms.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:cs_algorithms.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>lcs<br>valid_parentheses</details> | <details><summary>Description</summary><br>Solve two common computer-science tasks</details> | Multiple-Choice
[elementary_math_qa](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/elementary_math_qa/) | <details><summary>Tasks</summary><br>`bigbench:elementary_math_qa.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:elementary_math_qa.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:elementary_math_qa.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:elementary_math_qa.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:elementary_math_qa.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:elementary_math_qa.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:elementary_math_qa.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:elementary_math_qa.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>language_hint_only<br>mathematical_hint_only<br>question_only<br>question_with_language_hint<br>question_with_mathematical_hint</details> | <details><summary>Description</summary><br>Answer multiple choice mathematical word problems</details> | Multiple-Choice
[fact_checker](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/fact_checker/) | <details><summary>Tasks</summary><br>`bigbench:fact_checker.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:fact_checker.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:fact_checker.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:fact_checker.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:fact_checker.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:fact_checker.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:fact_checker.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:fact_checker.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>covid19_scientific<br>fever<br>politifact</details> | <details><summary>Description</summary><br>Evaluate claims as true or false</details> | Multiple-Choice
[gem](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/gem/) | <details><summary>Tasks</summary><br>`bigbench:gem.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.78_examples.asset`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.78_examples.common_gen`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.78_examples.cs_restaurants`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.78_examples.e2e_nlg`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.78_examples.mlsum_de`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.78_examples.mlsum_es`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.78_examples.schema_guided_dialog`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.78_examples.turk`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.78_examples.webnlg_en`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.78_examples.webnlg_ru`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.78_examples.wikilingua_en`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.78_examples.wikilingua_multilingual`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.78_examples.xsum`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.all_examples.asset`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.all_examples.common_gen`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.all_examples.cs_restaurants`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.all_examples.e2e_nlg`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.all_examples.mlsum_de`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.all_examples.mlsum_es`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.all_examples.schema_guided_dialog`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.all_examples.turk`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.all_examples.webnlg_en`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.all_examples.webnlg_ru`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.all_examples.wikilingua_en`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.all_examples.wikilingua_multilingual`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.all_examples.xsum`<br>`bigbench:gem.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.78_examples.asset`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.78_examples.common_gen`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.78_examples.cs_restaurants`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.78_examples.e2e_nlg`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.78_examples.mlsum_de`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.78_examples.mlsum_es`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.78_examples.schema_guided_dialog`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.78_examples.turk`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.78_examples.webnlg_en`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.78_examples.webnlg_ru`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.78_examples.wikilingua_en`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.78_examples.wikilingua_multilingual`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.78_examples.xsum`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.all_examples.asset`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.all_examples.common_gen`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.all_examples.cs_restaurants`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.all_examples.e2e_nlg`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.all_examples.mlsum_de`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.all_examples.mlsum_es`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.all_examples.schema_guided_dialog`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.all_examples.turk`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.all_examples.webnlg_en`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.all_examples.webnlg_ru`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.all_examples.wikilingua_en`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.all_examples.wikilingua_multilingual`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.all_examples.xsum`<br>`bigbench:gem.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.78_examples.asset`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.78_examples.common_gen`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.78_examples.cs_restaurants`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.78_examples.e2e_nlg`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.78_examples.mlsum_de`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.78_examples.mlsum_es`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.78_examples.schema_guided_dialog`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.78_examples.turk`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.78_examples.webnlg_en`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.78_examples.webnlg_ru`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.78_examples.wikilingua_en`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.78_examples.wikilingua_multilingual`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.78_examples.xsum`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.all_examples.asset`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.all_examples.common_gen`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.all_examples.cs_restaurants`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.all_examples.e2e_nlg`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.all_examples.mlsum_de`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.all_examples.mlsum_es`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.all_examples.schema_guided_dialog`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.all_examples.turk`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.all_examples.webnlg_en`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.all_examples.webnlg_ru`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.all_examples.wikilingua_en`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.all_examples.wikilingua_multilingual`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.all_examples.xsum`<br>`bigbench:gem.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.78_examples.asset`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.78_examples.common_gen`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.78_examples.cs_restaurants`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.78_examples.e2e_nlg`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.78_examples.mlsum_de`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.78_examples.mlsum_es`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.78_examples.schema_guided_dialog`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.78_examples.turk`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.78_examples.webnlg_en`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.78_examples.webnlg_ru`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.78_examples.wikilingua_en`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.78_examples.wikilingua_multilingual`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.78_examples.xsum`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.all_examples.asset`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.all_examples.common_gen`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.all_examples.cs_restaurants`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.all_examples.e2e_nlg`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.all_examples.mlsum_de`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.all_examples.mlsum_es`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.all_examples.schema_guided_dialog`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.all_examples.turk`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.all_examples.webnlg_en`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.all_examples.webnlg_ru`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.all_examples.wikilingua_en`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.all_examples.wikilingua_multilingual`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.all_examples.xsum`<br>`bigbench:gem.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>asset<br>common_gen<br>cs_restaurants<br>e2e_nlg<br>mlsum_de<br>mlsum_es<br>schema_guided_dialog<br>turk<br>webnlg_en<br>webnlg_ru<br>wikilingua_en<br>wikilingua_multilingual<br>xsum</details> | <details><summary>Description</summary><br>The datasets included in this collection were modified from their original version as part of GEM to improve data quality or make them more challenging</details> | Generative, Scoring
[goal_step_wikihow](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/goal_step_wikihow/) | <details><summary>Tasks</summary><br>`bigbench:goal_step_wikihow.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:goal_step_wikihow.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:goal_step_wikihow.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:goal_step_wikihow.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:goal_step_wikihow.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:goal_step_wikihow.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:goal_step_wikihow.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:goal_step_wikihow.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>goal_inference<br>step_inference<br>step_ordering</details> | <details><summary>Description</summary><br>Perform one of three subtasks: step inference, goal inference, or step ordering</details> | Multiple-Choice
[hhh_alignment](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/hhh_alignment/) | <details><summary>Tasks</summary><br>`bigbench:hhh_alignment.mix.t5_default_vocab.0_shot.221_examples`<br>`bigbench:hhh_alignment.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:hhh_alignment.mix.t5_default_vocab.1_shot.221_examples`<br>`bigbench:hhh_alignment.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:hhh_alignment.mix.t5_default_vocab.2_shot.221_examples`<br>`bigbench:hhh_alignment.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:hhh_alignment.mix.t5_default_vocab.3_shot.221_examples`<br>`bigbench:hhh_alignment.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>Harms<br>Helpfulness<br>Honesty<br>Other</details> | <details><summary>Description</summary><br>Evaluate how helpful, honest, and harmless model responses are, when presented with requests or scenarios that probe model alignment</details> | Multiple-Choice
[intersect_geometry](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/intersect_geometry/) | <details><summary>Tasks</summary><br>`bigbench:intersect_geometry.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:intersect_geometry.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:intersect_geometry.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:intersect_geometry.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:intersect_geometry.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:intersect_geometry.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:intersect_geometry.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:intersect_geometry.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>shapes_2<br>shapes_3<br>shapes_4<br>shapes_5<br>shapes_6</details> | <details><summary>Description</summary><br>Find the number of intersection points between the shapes and lines specified by the given coordinates</details> | Multiple-Choice
[kanji_ascii](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/kanji_ascii/) | <details><summary>Tasks</summary><br>`bigbench:kanji_ascii.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:kanji_ascii.mix.t5_default_vocab.0_shot.512_examples.kanji_ascii_pronunciation`<br>`bigbench:kanji_ascii.mix.t5_default_vocab.0_shot.all_examples.kanji_ascii_pronunciation`<br>`bigbench:kanji_ascii.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:kanji_ascii.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:kanji_ascii.mix.t5_default_vocab.1_shot.512_examples.kanji_ascii_pronunciation`<br>`bigbench:kanji_ascii.mix.t5_default_vocab.1_shot.all_examples.kanji_ascii_pronunciation`<br>`bigbench:kanji_ascii.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:kanji_ascii.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:kanji_ascii.mix.t5_default_vocab.2_shot.512_examples.kanji_ascii_pronunciation`<br>`bigbench:kanji_ascii.mix.t5_default_vocab.2_shot.all_examples.kanji_ascii_pronunciation`<br>`bigbench:kanji_ascii.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:kanji_ascii.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:kanji_ascii.mix.t5_default_vocab.3_shot.512_examples.kanji_ascii_pronunciation`<br>`bigbench:kanji_ascii.mix.t5_default_vocab.3_shot.all_examples.kanji_ascii_pronunciation`<br>`bigbench:kanji_ascii.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>kanji_ascii_meaning<br>kanji_ascii_pronunciation</details> | <details><summary>Description</summary><br>Identify an object using the ASCII arts of various kanji</details> | Generative, Multiple-Choice, Scoring
[key_value_maps](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/key_value_maps/) | <details><summary>Tasks</summary><br>`bigbench:key_value_maps.mix.t5_default_vocab.0_shot.101_examples`<br>`bigbench:key_value_maps.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:key_value_maps.mix.t5_default_vocab.1_shot.101_examples`<br>`bigbench:key_value_maps.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:key_value_maps.mix.t5_default_vocab.2_shot.101_examples`<br>`bigbench:key_value_maps.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:key_value_maps.mix.t5_default_vocab.3_shot.101_examples`<br>`bigbench:key_value_maps.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>extends<br>extends_agree_on<br>extends_agree_on_only_differ<br>extends_undef_on_only_differ</details> | <details><summary>Description</summary><br>Decide the truth of formal statements about key/value maps</details> | Multiple-Choice
[language_games](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/language_games/) | <details><summary>Tasks</summary><br>`bigbench:language_games.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:language_games.mix.t5_default_vocab.0_shot.170_examples.pig_latin_decode`<br>`bigbench:language_games.mix.t5_default_vocab.0_shot.170_examples.pig_latin_encode`<br>`bigbench:language_games.mix.t5_default_vocab.0_shot.32_examples.egg_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.0_shot.32_examples.egg_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.0_shot.32_examples.pig_latin_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.0_shot.32_examples.pig_latin_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.0_shot.all_examples.egg_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.0_shot.all_examples.egg_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.0_shot.all_examples.pig_latin_decode`<br>`bigbench:language_games.mix.t5_default_vocab.0_shot.all_examples.pig_latin_encode`<br>`bigbench:language_games.mix.t5_default_vocab.0_shot.all_examples.pig_latin_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.0_shot.all_examples.pig_latin_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:language_games.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:language_games.mix.t5_default_vocab.1_shot.170_examples.pig_latin_decode`<br>`bigbench:language_games.mix.t5_default_vocab.1_shot.170_examples.pig_latin_encode`<br>`bigbench:language_games.mix.t5_default_vocab.1_shot.32_examples.egg_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.1_shot.32_examples.egg_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.1_shot.32_examples.pig_latin_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.1_shot.32_examples.pig_latin_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.1_shot.all_examples.egg_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.1_shot.all_examples.egg_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.1_shot.all_examples.pig_latin_decode`<br>`bigbench:language_games.mix.t5_default_vocab.1_shot.all_examples.pig_latin_encode`<br>`bigbench:language_games.mix.t5_default_vocab.1_shot.all_examples.pig_latin_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.1_shot.all_examples.pig_latin_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:language_games.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:language_games.mix.t5_default_vocab.2_shot.170_examples.pig_latin_decode`<br>`bigbench:language_games.mix.t5_default_vocab.2_shot.170_examples.pig_latin_encode`<br>`bigbench:language_games.mix.t5_default_vocab.2_shot.32_examples.egg_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.2_shot.32_examples.egg_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.2_shot.32_examples.pig_latin_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.2_shot.32_examples.pig_latin_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.2_shot.all_examples.egg_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.2_shot.all_examples.egg_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.2_shot.all_examples.pig_latin_decode`<br>`bigbench:language_games.mix.t5_default_vocab.2_shot.all_examples.pig_latin_encode`<br>`bigbench:language_games.mix.t5_default_vocab.2_shot.all_examples.pig_latin_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.2_shot.all_examples.pig_latin_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:language_games.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:language_games.mix.t5_default_vocab.3_shot.170_examples.pig_latin_decode`<br>`bigbench:language_games.mix.t5_default_vocab.3_shot.170_examples.pig_latin_encode`<br>`bigbench:language_games.mix.t5_default_vocab.3_shot.32_examples.egg_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.3_shot.32_examples.egg_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.3_shot.32_examples.pig_latin_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.3_shot.32_examples.pig_latin_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.3_shot.all_examples.egg_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.3_shot.all_examples.egg_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.3_shot.all_examples.pig_latin_decode`<br>`bigbench:language_games.mix.t5_default_vocab.3_shot.all_examples.pig_latin_encode`<br>`bigbench:language_games.mix.t5_default_vocab.3_shot.all_examples.pig_latin_encoded_target`<br>`bigbench:language_games.mix.t5_default_vocab.3_shot.all_examples.pig_latin_plain_target`<br>`bigbench:language_games.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>egg_encoded_target<br>egg_plain_target<br>pig_latin_decode<br>pig_latin_encode<br>pig_latin_encoded_target<br>pig_latin_plain_target</details> | <details><summary>Description</summary><br>Play language games, eg. translate between pig Latin and English, or responde to statements in pig Latin or English.</details> | Generative, Scoring
[linguistic_mappings](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/linguistic_mappings/) | <details><summary>Tasks</summary><br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.32_examples.plural_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.85_examples.de_past_tense_irregular`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.85_examples.de_past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.85_examples.de_past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.85_examples.past_tense_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.85_examples.past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.85_examples.past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.85_examples.plural_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.85_examples.plural_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.85_examples.pronoun_replacement_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.85_examples.question_formation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.85_examples.sentence_negation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.all_examples.de_past_tense_irregular`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.all_examples.de_past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.all_examples.de_past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.all_examples.past_tense_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.all_examples.past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.all_examples.past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.all_examples.plural_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.all_examples.plural_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.all_examples.plural_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.all_examples.pronoun_replacement_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.all_examples.question_formation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.all_examples.sentence_negation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.32_examples.plural_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.85_examples.de_past_tense_irregular`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.85_examples.de_past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.85_examples.de_past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.85_examples.past_tense_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.85_examples.past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.85_examples.past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.85_examples.plural_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.85_examples.plural_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.85_examples.pronoun_replacement_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.85_examples.question_formation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.85_examples.sentence_negation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.all_examples.de_past_tense_irregular`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.all_examples.de_past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.all_examples.de_past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.all_examples.past_tense_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.all_examples.past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.all_examples.past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.all_examples.plural_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.all_examples.plural_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.all_examples.plural_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.all_examples.pronoun_replacement_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.all_examples.question_formation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.all_examples.sentence_negation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.32_examples.plural_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.85_examples.de_past_tense_irregular`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.85_examples.de_past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.85_examples.de_past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.85_examples.past_tense_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.85_examples.past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.85_examples.past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.85_examples.plural_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.85_examples.plural_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.85_examples.pronoun_replacement_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.85_examples.question_formation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.85_examples.sentence_negation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.all_examples.de_past_tense_irregular`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.all_examples.de_past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.all_examples.de_past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.all_examples.past_tense_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.all_examples.past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.all_examples.past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.all_examples.plural_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.all_examples.plural_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.all_examples.plural_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.all_examples.pronoun_replacement_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.all_examples.question_formation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.all_examples.sentence_negation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.32_examples.plural_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.85_examples.de_past_tense_irregular`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.85_examples.de_past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.85_examples.de_past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.85_examples.past_tense_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.85_examples.past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.85_examples.past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.85_examples.plural_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.85_examples.plural_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.85_examples.pronoun_replacement_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.85_examples.question_formation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.85_examples.sentence_negation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.all_examples.de_past_tense_irregular`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.all_examples.de_past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.all_examples.de_past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.all_examples.past_tense_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.all_examples.past_tense_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.all_examples.past_tense_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.all_examples.plural_irregular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.all_examples.plural_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.all_examples.plural_regular_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.all_examples.pronoun_replacement_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.all_examples.question_formation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.all_examples.sentence_negation_json`<br>`bigbench:linguistic_mappings.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>de_past_tense_irregular<br>de_past_tense_json<br>de_past_tense_regular_json<br>past_tense_irregular_json<br>past_tense_json<br>past_tense_regular_json<br>plural_irregular_json<br>plural_json<br>plural_regular_json<br>pronoun_replacement_json<br>question_formation_json<br>sentence_negation_json</details> | <details><summary>Description</summary><br>Use grammatical abstractions for morphological and syntactic linguistic mappings in fewshot learning</details> | Generative, Scoring
[list_functions](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/list_functions/) | <details><summary>Tasks</summary><br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.00_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.01_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.02_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.03_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.04_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.05_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.06_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.07_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.08_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.09_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.10_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c001`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c002`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c003`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c004`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c005`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c006`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c007`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c008`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c009`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c010`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c011`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c012`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c013`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c014`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c015`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c016`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c017`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c018`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c019`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c020`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c021`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c022`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c023`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c024`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c025`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c026`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c027`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c028`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c029`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c030`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c031`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c032`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c033`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c034`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c035`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c036`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c037`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c038`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c039`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c040`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c041`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c042`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c043`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c044`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c045`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c046`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c047`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c048`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c049`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c050`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c051`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c052`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c053`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c054`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c055`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c056`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c057`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c058`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c059`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c060`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c061`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c062`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c063`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c064`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c065`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c066`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c067`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c068`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c069`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c070`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c071`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c072`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c073`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c074`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c075`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c076`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c077`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c078`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c079`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c080`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c081`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c082`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c083`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c084`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c085`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c086`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c087`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c088`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c089`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c090`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c091`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c092`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c093`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c094`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c095`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c096`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c097`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c098`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c099`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c100`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c101`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c102`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c103`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c104`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c105`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c106`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c107`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c108`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c109`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c110`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c111`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c112`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c113`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c114`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c115`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c116`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c117`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c118`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c119`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c120`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c121`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c122`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c123`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c124`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c125`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c126`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c127`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c128`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c129`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c130`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c131`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c132`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c133`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c134`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c135`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c136`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c137`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c138`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c139`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c140`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c141`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c142`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c143`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c144`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c145`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c146`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c147`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c148`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c149`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c150`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c151`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c152`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c153`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c154`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c155`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c156`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c157`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c158`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c159`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c160`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c161`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c162`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c163`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c164`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c165`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c166`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c167`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c168`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c169`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c170`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c171`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c172`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c173`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c174`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c175`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c176`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c177`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c178`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c179`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c180`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c181`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c182`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c183`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c184`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c185`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c186`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c187`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c188`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c189`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c190`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c191`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c192`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c193`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c194`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c195`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c196`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c197`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c198`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c199`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c200`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c201`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c202`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c203`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c204`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c205`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c206`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c207`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c208`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c209`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c210`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c211`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c212`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c213`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c214`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c215`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c216`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c217`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c218`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c219`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c220`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c221`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c222`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c223`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c224`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c225`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c226`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c227`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c228`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c229`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c230`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c231`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c232`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c233`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c234`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c235`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c236`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c237`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c238`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c239`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c240`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c241`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c242`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c243`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c244`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c245`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c246`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c247`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c248`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c249`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.3_examples.c250`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.00_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.01_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.02_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.03_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.04_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.05_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.06_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.07_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.08_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.09_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.10_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c001`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c002`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c003`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c004`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c005`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c006`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c007`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c008`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c009`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c010`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c011`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c012`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c013`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c014`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c015`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c016`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c017`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c018`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c019`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c020`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c021`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c022`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c023`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c024`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c025`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c026`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c027`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c028`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c029`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c030`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c031`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c032`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c033`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c034`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c035`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c036`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c037`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c038`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c039`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c040`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c041`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c042`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c043`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c044`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c045`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c046`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c047`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c048`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c049`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c050`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c051`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c052`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c053`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c054`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c055`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c056`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c057`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c058`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c059`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c060`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c061`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c062`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c063`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c064`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c065`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c066`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c067`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c068`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c069`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c070`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c071`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c072`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c073`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c074`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c075`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c076`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c077`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c078`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c079`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c080`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c081`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c082`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c083`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c084`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c085`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c086`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c087`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c088`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c089`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c090`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c091`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c092`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c093`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c094`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c095`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c096`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c097`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c098`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c099`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c100`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c101`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c102`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c103`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c104`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c105`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c106`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c107`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c108`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c109`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c110`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c111`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c112`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c113`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c114`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c115`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c116`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c117`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c118`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c119`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c120`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c121`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c122`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c123`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c124`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c125`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c126`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c127`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c128`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c129`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c130`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c131`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c132`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c133`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c134`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c135`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c136`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c137`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c138`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c139`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c140`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c141`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c142`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c143`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c144`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c145`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c146`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c147`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c148`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c149`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c150`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c151`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c152`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c153`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c154`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c155`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c156`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c157`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c158`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c159`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c160`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c161`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c162`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c163`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c164`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c165`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c166`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c167`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c168`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c169`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c170`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c171`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c172`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c173`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c174`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c175`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c176`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c177`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c178`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c179`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c180`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c181`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c182`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c183`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c184`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c185`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c186`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c187`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c188`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c189`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c190`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c191`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c192`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c193`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c194`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c195`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c196`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c197`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c198`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c199`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c200`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c201`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c202`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c203`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c204`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c205`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c206`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c207`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c208`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c209`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c210`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c211`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c212`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c213`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c214`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c215`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c216`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c217`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c218`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c219`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c220`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c221`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c222`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c223`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c224`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c225`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c226`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c227`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c228`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c229`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c230`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c231`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c232`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c233`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c234`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c235`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c236`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c237`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c238`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c239`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c240`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c241`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c242`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c243`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c244`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c245`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c246`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c247`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c248`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c249`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples.c250`<br>`bigbench:list_functions.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.00_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.01_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.02_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.03_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.04_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.05_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.06_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.07_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.08_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.09_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.10_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c001`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c002`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c003`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c004`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c005`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c006`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c007`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c008`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c009`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c010`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c011`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c012`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c013`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c014`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c015`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c016`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c017`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c018`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c019`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c020`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c021`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c022`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c023`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c024`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c025`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c026`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c027`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c028`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c029`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c030`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c031`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c032`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c033`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c034`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c035`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c036`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c037`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c038`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c039`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c040`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c041`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c042`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c043`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c044`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c045`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c046`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c047`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c048`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c049`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c050`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c051`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c052`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c053`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c054`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c055`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c056`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c057`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c058`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c059`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c060`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c061`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c062`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c063`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c064`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c065`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c066`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c067`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c068`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c069`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c070`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c071`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c072`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c073`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c074`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c075`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c076`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c077`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c078`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c079`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c080`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c081`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c082`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c083`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c084`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c085`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c086`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c087`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c088`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c089`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c090`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c091`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c092`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c093`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c094`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c095`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c096`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c097`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c098`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c099`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c100`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c101`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c102`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c103`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c104`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c105`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c106`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c107`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c108`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c109`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c110`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c111`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c112`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c113`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c114`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c115`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c116`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c117`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c118`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c119`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c120`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c121`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c122`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c123`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c124`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c125`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c126`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c127`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c128`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c129`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c130`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c131`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c132`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c133`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c134`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c135`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c136`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c137`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c138`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c139`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c140`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c141`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c142`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c143`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c144`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c145`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c146`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c147`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c148`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c149`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c150`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c151`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c152`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c153`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c154`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c155`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c156`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c157`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c158`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c159`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c160`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c161`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c162`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c163`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c164`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c165`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c166`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c167`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c168`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c169`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c170`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c171`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c172`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c173`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c174`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c175`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c176`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c177`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c178`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c179`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c180`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c181`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c182`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c183`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c184`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c185`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c186`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c187`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c188`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c189`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c190`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c191`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c192`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c193`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c194`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c195`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c196`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c197`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c198`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c199`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c200`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c201`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c202`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c203`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c204`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c205`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c206`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c207`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c208`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c209`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c210`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c211`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c212`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c213`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c214`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c215`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c216`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c217`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c218`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c219`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c220`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c221`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c222`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c223`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c224`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c225`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c226`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c227`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c228`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c229`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c230`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c231`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c232`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c233`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c234`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c235`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c236`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c237`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c238`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c239`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c240`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c241`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c242`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c243`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c244`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c245`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c246`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c247`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c248`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c249`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.3_examples.c250`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.00_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.01_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.02_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.03_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.04_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.05_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.06_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.07_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.08_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.09_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.10_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c001`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c002`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c003`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c004`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c005`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c006`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c007`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c008`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c009`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c010`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c011`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c012`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c013`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c014`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c015`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c016`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c017`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c018`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c019`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c020`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c021`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c022`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c023`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c024`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c025`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c026`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c027`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c028`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c029`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c030`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c031`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c032`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c033`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c034`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c035`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c036`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c037`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c038`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c039`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c040`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c041`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c042`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c043`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c044`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c045`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c046`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c047`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c048`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c049`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c050`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c051`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c052`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c053`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c054`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c055`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c056`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c057`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c058`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c059`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c060`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c061`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c062`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c063`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c064`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c065`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c066`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c067`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c068`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c069`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c070`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c071`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c072`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c073`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c074`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c075`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c076`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c077`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c078`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c079`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c080`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c081`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c082`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c083`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c084`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c085`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c086`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c087`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c088`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c089`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c090`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c091`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c092`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c093`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c094`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c095`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c096`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c097`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c098`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c099`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c100`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c101`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c102`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c103`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c104`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c105`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c106`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c107`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c108`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c109`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c110`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c111`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c112`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c113`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c114`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c115`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c116`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c117`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c118`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c119`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c120`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c121`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c122`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c123`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c124`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c125`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c126`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c127`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c128`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c129`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c130`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c131`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c132`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c133`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c134`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c135`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c136`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c137`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c138`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c139`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c140`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c141`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c142`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c143`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c144`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c145`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c146`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c147`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c148`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c149`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c150`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c151`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c152`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c153`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c154`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c155`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c156`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c157`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c158`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c159`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c160`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c161`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c162`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c163`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c164`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c165`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c166`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c167`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c168`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c169`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c170`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c171`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c172`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c173`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c174`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c175`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c176`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c177`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c178`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c179`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c180`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c181`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c182`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c183`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c184`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c185`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c186`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c187`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c188`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c189`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c190`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c191`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c192`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c193`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c194`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c195`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c196`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c197`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c198`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c199`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c200`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c201`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c202`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c203`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c204`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c205`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c206`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c207`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c208`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c209`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c210`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c211`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c212`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c213`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c214`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c215`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c216`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c217`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c218`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c219`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c220`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c221`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c222`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c223`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c224`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c225`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c226`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c227`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c228`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c229`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c230`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c231`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c232`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c233`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c234`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c235`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c236`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c237`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c238`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c239`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c240`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c241`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c242`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c243`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c244`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c245`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c246`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c247`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c248`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c249`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples.c250`<br>`bigbench:list_functions.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.00_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.01_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.02_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.03_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.04_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.05_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.06_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.07_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.08_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.09_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.10_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c001`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c002`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c003`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c004`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c005`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c006`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c007`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c008`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c009`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c010`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c011`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c012`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c013`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c014`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c015`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c016`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c017`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c018`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c019`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c020`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c021`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c022`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c023`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c024`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c025`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c026`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c027`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c028`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c029`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c030`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c031`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c032`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c033`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c034`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c035`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c036`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c037`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c038`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c039`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c040`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c041`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c042`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c043`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c044`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c045`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c046`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c047`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c048`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c049`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c050`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c051`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c052`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c053`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c054`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c055`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c056`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c057`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c058`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c059`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c060`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c061`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c062`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c063`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c064`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c065`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c066`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c067`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c068`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c069`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c070`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c071`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c072`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c073`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c074`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c075`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c076`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c077`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c078`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c079`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c080`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c081`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c082`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c083`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c084`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c085`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c086`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c087`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c088`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c089`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c090`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c091`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c092`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c093`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c094`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c095`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c096`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c097`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c098`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c099`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c100`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c101`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c102`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c103`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c104`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c105`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c106`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c107`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c108`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c109`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c110`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c111`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c112`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c113`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c114`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c115`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c116`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c117`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c118`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c119`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c120`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c121`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c122`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c123`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c124`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c125`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c126`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c127`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c128`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c129`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c130`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c131`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c132`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c133`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c134`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c135`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c136`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c137`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c138`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c139`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c140`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c141`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c142`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c143`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c144`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c145`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c146`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c147`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c148`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c149`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c150`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c151`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c152`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c153`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c154`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c155`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c156`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c157`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c158`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c159`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c160`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c161`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c162`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c163`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c164`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c165`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c166`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c167`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c168`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c169`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c170`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c171`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c172`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c173`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c174`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c175`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c176`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c177`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c178`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c179`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c180`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c181`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c182`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c183`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c184`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c185`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c186`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c187`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c188`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c189`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c190`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c191`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c192`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c193`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c194`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c195`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c196`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c197`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c198`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c199`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c200`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c201`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c202`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c203`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c204`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c205`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c206`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c207`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c208`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c209`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c210`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c211`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c212`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c213`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c214`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c215`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c216`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c217`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c218`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c219`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c220`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c221`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c222`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c223`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c224`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c225`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c226`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c227`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c228`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c229`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c230`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c231`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c232`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c233`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c234`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c235`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c236`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c237`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c238`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c239`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c240`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c241`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c242`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c243`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c244`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c245`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c246`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c247`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c248`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c249`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.3_examples.c250`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.00_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.01_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.02_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.03_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.04_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.05_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.06_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.07_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.08_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.09_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.10_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c001`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c002`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c003`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c004`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c005`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c006`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c007`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c008`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c009`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c010`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c011`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c012`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c013`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c014`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c015`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c016`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c017`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c018`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c019`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c020`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c021`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c022`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c023`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c024`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c025`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c026`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c027`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c028`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c029`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c030`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c031`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c032`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c033`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c034`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c035`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c036`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c037`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c038`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c039`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c040`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c041`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c042`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c043`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c044`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c045`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c046`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c047`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c048`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c049`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c050`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c051`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c052`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c053`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c054`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c055`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c056`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c057`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c058`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c059`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c060`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c061`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c062`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c063`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c064`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c065`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c066`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c067`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c068`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c069`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c070`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c071`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c072`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c073`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c074`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c075`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c076`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c077`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c078`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c079`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c080`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c081`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c082`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c083`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c084`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c085`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c086`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c087`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c088`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c089`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c090`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c091`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c092`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c093`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c094`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c095`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c096`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c097`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c098`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c099`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c100`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c101`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c102`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c103`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c104`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c105`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c106`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c107`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c108`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c109`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c110`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c111`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c112`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c113`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c114`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c115`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c116`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c117`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c118`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c119`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c120`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c121`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c122`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c123`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c124`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c125`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c126`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c127`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c128`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c129`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c130`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c131`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c132`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c133`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c134`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c135`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c136`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c137`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c138`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c139`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c140`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c141`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c142`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c143`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c144`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c145`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c146`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c147`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c148`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c149`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c150`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c151`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c152`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c153`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c154`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c155`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c156`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c157`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c158`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c159`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c160`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c161`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c162`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c163`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c164`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c165`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c166`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c167`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c168`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c169`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c170`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c171`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c172`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c173`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c174`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c175`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c176`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c177`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c178`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c179`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c180`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c181`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c182`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c183`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c184`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c185`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c186`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c187`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c188`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c189`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c190`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c191`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c192`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c193`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c194`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c195`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c196`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c197`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c198`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c199`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c200`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c201`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c202`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c203`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c204`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c205`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c206`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c207`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c208`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c209`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c210`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c211`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c212`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c213`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c214`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c215`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c216`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c217`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c218`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c219`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c220`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c221`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c222`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c223`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c224`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c225`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c226`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c227`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c228`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c229`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c230`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c231`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c232`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c233`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c234`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c235`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c236`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c237`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c238`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c239`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c240`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c241`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c242`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c243`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c244`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c245`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c246`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c247`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c248`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c249`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples.c250`<br>`bigbench:list_functions.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.00_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.01_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.02_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.03_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.04_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.05_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.06_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.07_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.08_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.09_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.10_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c001`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c002`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c003`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c004`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c005`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c006`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c007`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c008`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c009`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c010`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c011`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c012`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c013`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c014`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c015`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c016`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c017`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c018`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c019`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c020`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c021`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c022`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c023`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c024`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c025`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c026`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c027`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c028`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c029`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c030`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c031`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c032`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c033`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c034`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c035`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c036`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c037`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c038`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c039`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c040`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c041`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c042`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c043`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c044`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c045`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c046`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c047`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c048`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c049`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c050`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c051`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c052`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c053`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c054`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c055`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c056`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c057`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c058`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c059`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c060`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c061`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c062`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c063`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c064`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c065`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c066`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c067`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c068`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c069`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c070`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c071`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c072`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c073`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c074`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c075`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c076`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c077`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c078`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c079`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c080`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c081`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c082`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c083`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c084`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c085`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c086`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c087`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c088`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c089`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c090`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c091`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c092`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c093`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c094`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c095`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c096`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c097`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c098`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c099`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c100`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c101`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c102`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c103`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c104`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c105`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c106`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c107`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c108`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c109`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c110`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c111`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c112`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c113`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c114`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c115`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c116`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c117`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c118`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c119`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c120`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c121`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c122`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c123`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c124`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c125`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c126`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c127`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c128`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c129`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c130`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c131`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c132`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c133`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c134`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c135`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c136`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c137`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c138`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c139`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c140`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c141`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c142`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c143`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c144`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c145`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c146`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c147`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c148`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c149`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c150`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c151`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c152`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c153`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c154`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c155`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c156`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c157`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c158`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c159`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c160`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c161`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c162`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c163`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c164`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c165`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c166`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c167`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c168`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c169`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c170`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c171`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c172`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c173`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c174`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c175`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c176`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c177`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c178`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c179`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c180`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c181`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c182`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c183`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c184`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c185`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c186`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c187`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c188`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c189`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c190`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c191`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c192`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c193`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c194`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c195`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c196`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c197`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c198`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c199`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c200`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c201`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c202`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c203`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c204`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c205`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c206`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c207`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c208`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c209`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c210`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c211`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c212`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c213`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c214`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c215`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c216`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c217`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c218`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c219`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c220`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c221`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c222`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c223`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c224`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c225`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c226`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c227`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c228`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c229`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c230`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c231`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c232`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c233`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c234`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c235`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c236`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c237`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c238`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c239`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c240`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c241`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c242`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c243`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c244`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c245`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c246`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c247`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c248`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c249`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.3_examples.c250`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.00_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.01_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.02_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.03_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.04_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.05_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.06_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.07_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.08_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.09_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.10_shot_baseline`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c001`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c002`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c003`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c004`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c005`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c006`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c007`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c008`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c009`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c010`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c011`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c012`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c013`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c014`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c015`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c016`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c017`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c018`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c019`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c020`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c021`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c022`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c023`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c024`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c025`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c026`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c027`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c028`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c029`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c030`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c031`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c032`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c033`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c034`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c035`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c036`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c037`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c038`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c039`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c040`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c041`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c042`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c043`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c044`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c045`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c046`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c047`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c048`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c049`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c050`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c051`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c052`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c053`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c054`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c055`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c056`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c057`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c058`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c059`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c060`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c061`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c062`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c063`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c064`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c065`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c066`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c067`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c068`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c069`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c070`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c071`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c072`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c073`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c074`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c075`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c076`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c077`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c078`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c079`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c080`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c081`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c082`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c083`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c084`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c085`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c086`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c087`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c088`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c089`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c090`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c091`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c092`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c093`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c094`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c095`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c096`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c097`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c098`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c099`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c100`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c101`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c102`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c103`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c104`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c105`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c106`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c107`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c108`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c109`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c110`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c111`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c112`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c113`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c114`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c115`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c116`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c117`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c118`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c119`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c120`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c121`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c122`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c123`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c124`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c125`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c126`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c127`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c128`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c129`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c130`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c131`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c132`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c133`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c134`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c135`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c136`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c137`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c138`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c139`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c140`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c141`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c142`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c143`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c144`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c145`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c146`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c147`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c148`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c149`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c150`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c151`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c152`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c153`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c154`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c155`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c156`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c157`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c158`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c159`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c160`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c161`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c162`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c163`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c164`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c165`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c166`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c167`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c168`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c169`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c170`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c171`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c172`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c173`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c174`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c175`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c176`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c177`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c178`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c179`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c180`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c181`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c182`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c183`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c184`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c185`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c186`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c187`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c188`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c189`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c190`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c191`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c192`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c193`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c194`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c195`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c196`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c197`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c198`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c199`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c200`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c201`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c202`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c203`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c204`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c205`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c206`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c207`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c208`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c209`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c210`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c211`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c212`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c213`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c214`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c215`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c216`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c217`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c218`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c219`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c220`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c221`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c222`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c223`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c224`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c225`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c226`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c227`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c228`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c229`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c230`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c231`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c232`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c233`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c234`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c235`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c236`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c237`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c238`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c239`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c240`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c241`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c242`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c243`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c244`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c245`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c246`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c247`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c248`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c249`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples.c250`<br>`bigbench:list_functions.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>00_shot_baseline<br>01_shot_baseline<br>02_shot_baseline<br>03_shot_baseline<br>04_shot_baseline<br>05_shot_baseline<br>06_shot_baseline<br>07_shot_baseline<br>08_shot_baseline<br>09_shot_baseline<br>10_shot_baseline<br>c001<br>c002<br>c003<br>c004<br>c005<br>c006<br>c007<br>c008<br>c009<br>c010<br>c011<br>c012<br>c013<br>c014<br>c015<br>c016<br>c017<br>c018<br>c019<br>c020<br>c021<br>c022<br>c023<br>c024<br>c025<br>c026<br>c027<br>c028<br>c029<br>c030<br>c031<br>c032<br>c033<br>c034<br>c035<br>c036<br>c037<br>c038<br>c039<br>c040<br>c041<br>c042<br>c043<br>c044<br>c045<br>c046<br>c047<br>c048<br>c049<br>c050<br>c051<br>c052<br>c053<br>c054<br>c055<br>c056<br>c057<br>c058<br>c059<br>c060<br>c061<br>c062<br>c063<br>c064<br>c065<br>c066<br>c067<br>c068<br>c069<br>c070<br>c071<br>c072<br>c073<br>c074<br>c075<br>c076<br>c077<br>c078<br>c079<br>c080<br>c081<br>c082<br>c083<br>c084<br>c085<br>c086<br>c087<br>c088<br>c089<br>c090<br>c091<br>c092<br>c093<br>c094<br>c095<br>c096<br>c097<br>c098<br>c099<br>c100<br>c101<br>c102<br>c103<br>c104<br>c105<br>c106<br>c107<br>c108<br>c109<br>c110<br>c111<br>c112<br>c113<br>c114<br>c115<br>c116<br>c117<br>c118<br>c119<br>c120<br>c121<br>c122<br>c123<br>c124<br>c125<br>c126<br>c127<br>c128<br>c129<br>c130<br>c131<br>c132<br>c133<br>c134<br>c135<br>c136<br>c137<br>c138<br>c139<br>c140<br>c141<br>c142<br>c143<br>c144<br>c145<br>c146<br>c147<br>c148<br>c149<br>c150<br>c151<br>c152<br>c153<br>c154<br>c155<br>c156<br>c157<br>c158<br>c159<br>c160<br>c161<br>c162<br>c163<br>c164<br>c165<br>c166<br>c167<br>c168<br>c169<br>c170<br>c171<br>c172<br>c173<br>c174<br>c175<br>c176<br>c177<br>c178<br>c179<br>c180<br>c181<br>c182<br>c183<br>c184<br>c185<br>c186<br>c187<br>c188<br>c189<br>c190<br>c191<br>c192<br>c193<br>c194<br>c195<br>c196<br>c197<br>c198<br>c199<br>c200<br>c201<br>c202<br>c203<br>c204<br>c205<br>c206<br>c207<br>c208<br>c209<br>c210<br>c211<br>c212<br>c213<br>c214<br>c215<br>c216<br>c217<br>c218<br>c219<br>c220<br>c221<br>c222<br>c223<br>c224<br>c225<br>c226<br>c227<br>c228<br>c229<br>c230<br>c231<br>c232<br>c233<br>c234<br>c235<br>c236<br>c237<br>c238<br>c239<br>c240<br>c241<br>c242<br>c243<br>c244<br>c245<br>c246<br>c247<br>c248<br>c249<br>c250</details> | <details><summary>Description</summary><br>Infer and compute functions over lists of natural numbers</details> | Generative, Scoring
[logical_deduction](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/logical_deduction/) | <details><summary>Tasks</summary><br>`bigbench:logical_deduction.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:logical_deduction.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:logical_deduction.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:logical_deduction.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:logical_deduction.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:logical_deduction.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:logical_deduction.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:logical_deduction.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>five_objects<br>seven_objects<br>three_objects</details> | <details><summary>Description</summary><br>Deduce the order of a sequence of objects</details> | Multiple-Choice
[metaphor_understanding](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/metaphor_understanding/) | <details><summary>Tasks</summary><br>`bigbench:metaphor_understanding.mix.t5_default_vocab.0_shot.234_examples`<br>`bigbench:metaphor_understanding.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:metaphor_understanding.mix.t5_default_vocab.1_shot.234_examples`<br>`bigbench:metaphor_understanding.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:metaphor_understanding.mix.t5_default_vocab.2_shot.234_examples`<br>`bigbench:metaphor_understanding.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:metaphor_understanding.mix.t5_default_vocab.3_shot.234_examples`<br>`bigbench:metaphor_understanding.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>lit2met<br>met2lit</details> | <details><summary>Description</summary><br>Translate between literal and metaphoric sentences</details> | Multiple-Choice
[minute_mysteries_qa](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/minute_mysteries_qa/) | <details><summary>Tasks</summary><br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.0_shot.274_examples.freetext`<br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.0_shot.477_examples`<br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.0_shot.all_examples.freetext`<br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.1_shot.274_examples.freetext`<br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.1_shot.477_examples`<br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.1_shot.all_examples.freetext`<br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.2_shot.274_examples.freetext`<br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.2_shot.477_examples`<br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.2_shot.all_examples.freetext`<br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.3_shot.274_examples.freetext`<br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.3_shot.477_examples`<br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.3_shot.all_examples.freetext`<br>`bigbench:minute_mysteries_qa.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>freetext<br>multiplechoice</details> | <details><summary>Description</summary><br>Given short crime stories, identify the perpetrator and explain the reasoning behind the deduction</details> | Generative, Multiple-Choice, Scoring
[modified_arithmetic](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/modified_arithmetic/) | <details><summary>Tasks</summary><br>`bigbench:modified_arithmetic.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.0_shot.170_examples.three_digit_addition_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.0_shot.170_examples.three_digit_addition_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.0_shot.170_examples.three_digit_subtraction_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.0_shot.170_examples.three_digit_subtraction_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.0_shot.170_examples.two_digit_multiplication_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.0_shot.170_examples.two_digit_multiplication_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.0_shot.all_examples.three_digit_addition_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.0_shot.all_examples.three_digit_addition_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.0_shot.all_examples.three_digit_subtraction_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.0_shot.all_examples.three_digit_subtraction_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.0_shot.all_examples.two_digit_multiplication_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.0_shot.all_examples.two_digit_multiplication_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.1_shot.170_examples.three_digit_addition_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.1_shot.170_examples.three_digit_addition_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.1_shot.170_examples.three_digit_subtraction_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.1_shot.170_examples.three_digit_subtraction_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.1_shot.170_examples.two_digit_multiplication_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.1_shot.170_examples.two_digit_multiplication_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.1_shot.all_examples.three_digit_addition_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.1_shot.all_examples.three_digit_addition_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.1_shot.all_examples.three_digit_subtraction_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.1_shot.all_examples.three_digit_subtraction_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.1_shot.all_examples.two_digit_multiplication_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.1_shot.all_examples.two_digit_multiplication_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.2_shot.170_examples.three_digit_addition_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.2_shot.170_examples.three_digit_addition_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.2_shot.170_examples.three_digit_subtraction_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.2_shot.170_examples.three_digit_subtraction_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.2_shot.170_examples.two_digit_multiplication_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.2_shot.170_examples.two_digit_multiplication_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.2_shot.all_examples.three_digit_addition_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.2_shot.all_examples.three_digit_addition_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.2_shot.all_examples.three_digit_subtraction_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.2_shot.all_examples.three_digit_subtraction_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.2_shot.all_examples.two_digit_multiplication_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.2_shot.all_examples.two_digit_multiplication_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.3_shot.170_examples.three_digit_addition_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.3_shot.170_examples.three_digit_addition_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.3_shot.170_examples.three_digit_subtraction_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.3_shot.170_examples.three_digit_subtraction_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.3_shot.170_examples.two_digit_multiplication_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.3_shot.170_examples.two_digit_multiplication_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.3_shot.all_examples.three_digit_addition_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.3_shot.all_examples.three_digit_addition_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.3_shot.all_examples.three_digit_subtraction_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.3_shot.all_examples.three_digit_subtraction_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.3_shot.all_examples.two_digit_multiplication_control`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.3_shot.all_examples.two_digit_multiplication_plus_one`<br>`bigbench:modified_arithmetic.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>three_digit_addition_control<br>three_digit_addition_plus_one<br>three_digit_subtraction_control<br>three_digit_subtraction_plus_one<br>two_digit_multiplication_control<br>two_digit_multiplication_plus_one</details> | <details><summary>Description</summary><br>Given two three-digit numbers, perform an operation and add one</details> | Generative, Scoring
[mult_data_wrangling](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/mult_data_wrangling/) | <details><summary>Tasks</summary><br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.add_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.add_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.add_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.add_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.add_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.add_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.add_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.add_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.add_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.add_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.add_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.add_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.after_symbol_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.after_symbol_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.after_symbol_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.after_symbol_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.append_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.append_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.append_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.append_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.append_time_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.append_time_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.append_time_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.append_time_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.between_symbols_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.between_symbols_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.between_symbols_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.between_symbols_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.brackets_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.brackets_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.brackets_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.brackets_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.change_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.change_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.change_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.change_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.change_format_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.change_format_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.change_format_4_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.change_format_4_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.change_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.change_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.change_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.change_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_10_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_10_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_3_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_3_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_4_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_4_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_5_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_5_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_6_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_6_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_7_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_7_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_8_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_8_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_9_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.convert_9_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.country_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.country_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.country_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.country_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.country_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.country_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.country_prefix_7_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.country_prefix_7_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.country_prefix_8_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.country_prefix_8_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.country_prefix_9_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.country_prefix_9_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_parentheses_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_parentheses_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_parentheses_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_parentheses_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_punctuation_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_punctuation_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_punctuation_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_punctuation_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_punctuation_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_punctuation_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_punctuation_4_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_punctuation_4_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_spaces_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_spaces_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_spaces_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_spaces_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.delete_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.digit_to_end_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.digit_to_end_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.digit_to_end_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.digit_to_end_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.first_character_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.first_character_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.first_character_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.first_character_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.generate_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.generate_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.generate_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.generate_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.generate_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.generate_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_after_at_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_after_at_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_after_at_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_after_at_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_after_at_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_after_at_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_after_at_4_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_after_at_4_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_after_comma_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_after_comma_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_after_comma_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_after_comma_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_between_commas_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_between_commas_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_between_commas_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_between_commas_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_caps_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_caps_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_caps_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_caps_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_caps_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_caps_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_day_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_day_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_day_ordinal_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_day_ordinal_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_day_ordinal_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_day_ordinal_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_domain_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_domain_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_domain_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_domain_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_hour_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_hour_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_hour_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_hour_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_minutes_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_minutes_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_minutes_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_minutes_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_number_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_number_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_number_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_number_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_system_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_system_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_system_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_system_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_units_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_units_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_units_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_units_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_value_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_value_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_value_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_value_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_week_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_week_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_week_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.get_week_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.login_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.login_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.login_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.login_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_3_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_3_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_4_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_4_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_5_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_5_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_6_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_6_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_7_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_7_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_8_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_8_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_9_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.reduce_name_9_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_prefix_4_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_prefix_4_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_prefix_5_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_prefix_5_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_prefix_6_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_prefix_6_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_punctuation_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_punctuation_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_punctuation_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.set_punctuation_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.some_before_at_na_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.some_before_at_na_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.to_upper_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.to_upper_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.to_upper_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.4_examples.to_upper_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.add_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.add_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.add_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.add_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.add_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.add_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.add_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.add_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.add_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.add_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.add_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.add_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.after_symbol_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.after_symbol_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.after_symbol_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.after_symbol_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.append_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.append_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.append_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.append_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.append_time_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.append_time_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.append_time_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.append_time_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.between_symbols_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.between_symbols_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.between_symbols_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.between_symbols_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.brackets_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.brackets_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.brackets_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.brackets_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.change_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.change_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.change_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.change_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.change_format_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.change_format_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.change_format_4_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.change_format_4_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.change_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.change_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.change_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.change_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_10_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_10_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_3_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_3_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_4_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_4_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_5_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_5_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_6_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_6_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_7_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_7_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_8_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_8_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_9_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.convert_9_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.country_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.country_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.country_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.country_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.country_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.country_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.country_prefix_7_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.country_prefix_7_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.country_prefix_8_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.country_prefix_8_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.country_prefix_9_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.country_prefix_9_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_parentheses_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_parentheses_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_parentheses_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_parentheses_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_punctuation_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_punctuation_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_punctuation_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_punctuation_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_punctuation_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_punctuation_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_punctuation_4_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_punctuation_4_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_spaces_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_spaces_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_spaces_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_spaces_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.delete_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.digit_to_end_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.digit_to_end_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.digit_to_end_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.digit_to_end_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.first_character_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.first_character_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.first_character_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.first_character_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.generate_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.generate_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.generate_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.generate_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.generate_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.generate_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_after_at_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_after_at_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_after_at_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_after_at_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_after_at_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_after_at_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_after_at_4_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_after_at_4_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_after_comma_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_after_comma_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_after_comma_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_after_comma_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_between_commas_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_between_commas_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_between_commas_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_between_commas_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_caps_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_caps_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_caps_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_caps_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_caps_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_caps_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_day_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_day_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_day_ordinal_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_day_ordinal_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_day_ordinal_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_day_ordinal_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_domain_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_domain_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_domain_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_domain_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_hour_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_hour_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_hour_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_hour_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_minutes_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_minutes_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_minutes_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_minutes_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_number_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_number_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_number_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_number_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_system_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_system_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_system_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_system_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_units_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_units_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_units_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_units_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_value_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_value_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_value_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_value_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_week_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_week_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_week_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.get_week_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.login_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.login_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.login_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.login_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_3_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_3_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_4_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_4_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_5_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_5_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_6_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_6_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_7_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_7_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_8_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_8_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_9_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.reduce_name_9_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_prefix_4_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_prefix_4_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_prefix_5_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_prefix_5_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_prefix_6_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_prefix_6_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_punctuation_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_punctuation_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_punctuation_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.set_punctuation_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.some_before_at_na_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.some_before_at_na_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.to_upper_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.to_upper_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.to_upper_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples.to_upper_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.add_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.add_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.add_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.add_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.add_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.add_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.add_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.add_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.add_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.add_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.add_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.add_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.after_symbol_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.after_symbol_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.after_symbol_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.after_symbol_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.append_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.append_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.append_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.append_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.append_time_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.append_time_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.append_time_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.append_time_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.between_symbols_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.between_symbols_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.between_symbols_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.between_symbols_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.brackets_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.brackets_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.brackets_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.brackets_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.change_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.change_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.change_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.change_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.change_format_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.change_format_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.change_format_4_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.change_format_4_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.change_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.change_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.change_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.change_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_10_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_10_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_3_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_3_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_4_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_4_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_5_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_5_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_6_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_6_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_7_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_7_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_8_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_8_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_9_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.convert_9_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.country_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.country_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.country_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.country_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.country_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.country_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.country_prefix_7_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.country_prefix_7_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.country_prefix_8_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.country_prefix_8_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.country_prefix_9_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.country_prefix_9_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_parentheses_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_parentheses_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_parentheses_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_parentheses_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_punctuation_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_punctuation_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_punctuation_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_punctuation_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_punctuation_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_punctuation_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_punctuation_4_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_punctuation_4_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_spaces_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_spaces_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_spaces_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_spaces_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.delete_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.digit_to_end_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.digit_to_end_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.digit_to_end_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.digit_to_end_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.first_character_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.first_character_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.first_character_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.first_character_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.generate_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.generate_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.generate_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.generate_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.generate_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.generate_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_after_at_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_after_at_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_after_at_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_after_at_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_after_at_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_after_at_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_after_at_4_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_after_at_4_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_after_comma_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_after_comma_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_after_comma_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_after_comma_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_between_commas_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_between_commas_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_between_commas_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_between_commas_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_caps_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_caps_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_caps_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_caps_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_caps_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_caps_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_day_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_day_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_day_ordinal_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_day_ordinal_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_day_ordinal_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_day_ordinal_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_domain_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_domain_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_domain_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_domain_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_hour_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_hour_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_hour_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_hour_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_minutes_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_minutes_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_minutes_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_minutes_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_number_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_number_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_number_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_number_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_system_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_system_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_system_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_system_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_units_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_units_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_units_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_units_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_value_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_value_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_value_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_value_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_week_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_week_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_week_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.get_week_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.login_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.login_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.login_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.login_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_3_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_3_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_4_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_4_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_5_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_5_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_6_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_6_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_7_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_7_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_8_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_8_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_9_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.reduce_name_9_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_prefix_4_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_prefix_4_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_prefix_5_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_prefix_5_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_prefix_6_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_prefix_6_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_punctuation_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_punctuation_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_punctuation_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.set_punctuation_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.some_before_at_na_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.some_before_at_na_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.to_upper_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.to_upper_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.to_upper_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.4_examples.to_upper_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.add_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.add_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.add_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.add_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.add_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.add_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.add_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.add_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.add_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.add_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.add_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.add_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.after_symbol_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.after_symbol_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.after_symbol_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.after_symbol_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.append_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.append_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.append_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.append_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.append_time_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.append_time_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.append_time_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.append_time_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.between_symbols_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.between_symbols_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.between_symbols_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.between_symbols_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.brackets_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.brackets_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.brackets_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.brackets_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.change_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.change_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.change_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.change_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.change_format_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.change_format_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.change_format_4_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.change_format_4_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.change_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.change_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.change_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.change_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_10_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_10_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_3_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_3_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_4_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_4_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_5_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_5_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_6_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_6_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_7_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_7_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_8_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_8_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_9_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.convert_9_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.country_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.country_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.country_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.country_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.country_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.country_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.country_prefix_7_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.country_prefix_7_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.country_prefix_8_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.country_prefix_8_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.country_prefix_9_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.country_prefix_9_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_parentheses_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_parentheses_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_parentheses_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_parentheses_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_punctuation_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_punctuation_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_punctuation_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_punctuation_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_punctuation_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_punctuation_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_punctuation_4_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_punctuation_4_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_spaces_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_spaces_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_spaces_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_spaces_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.delete_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.digit_to_end_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.digit_to_end_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.digit_to_end_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.digit_to_end_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.first_character_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.first_character_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.first_character_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.first_character_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.generate_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.generate_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.generate_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.generate_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.generate_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.generate_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_after_at_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_after_at_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_after_at_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_after_at_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_after_at_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_after_at_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_after_at_4_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_after_at_4_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_after_comma_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_after_comma_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_after_comma_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_after_comma_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_between_commas_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_between_commas_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_between_commas_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_between_commas_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_caps_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_caps_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_caps_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_caps_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_caps_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_caps_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_day_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_day_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_day_ordinal_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_day_ordinal_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_day_ordinal_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_day_ordinal_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_domain_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_domain_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_domain_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_domain_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_hour_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_hour_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_hour_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_hour_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_minutes_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_minutes_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_minutes_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_minutes_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_number_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_number_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_number_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_number_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_system_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_system_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_system_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_system_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_units_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_units_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_units_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_units_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_value_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_value_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_value_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_value_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_week_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_week_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_week_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.get_week_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.login_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.login_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.login_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.login_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_3_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_3_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_4_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_4_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_5_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_5_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_6_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_6_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_7_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_7_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_8_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_8_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_9_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.reduce_name_9_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_prefix_4_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_prefix_4_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_prefix_5_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_prefix_5_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_prefix_6_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_prefix_6_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_punctuation_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_punctuation_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_punctuation_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.set_punctuation_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.some_before_at_na_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.some_before_at_na_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.to_upper_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.to_upper_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.to_upper_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples.to_upper_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.add_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.add_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.add_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.add_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.add_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.add_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.add_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.add_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.add_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.add_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.add_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.add_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.after_symbol_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.after_symbol_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.after_symbol_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.after_symbol_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.append_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.append_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.append_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.append_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.append_time_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.append_time_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.append_time_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.append_time_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.between_symbols_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.between_symbols_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.between_symbols_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.between_symbols_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.brackets_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.brackets_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.brackets_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.brackets_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.change_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.change_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.change_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.change_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.change_format_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.change_format_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.change_format_4_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.change_format_4_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.change_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.change_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.change_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.change_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_10_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_10_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_3_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_3_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_4_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_4_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_5_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_5_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_6_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_6_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_7_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_7_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_8_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_8_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_9_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.convert_9_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.country_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.country_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.country_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.country_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.country_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.country_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.country_prefix_7_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.country_prefix_7_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.country_prefix_8_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.country_prefix_8_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.country_prefix_9_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.country_prefix_9_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_parentheses_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_parentheses_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_parentheses_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_parentheses_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_punctuation_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_punctuation_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_punctuation_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_punctuation_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_punctuation_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_punctuation_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_punctuation_4_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_punctuation_4_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_spaces_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_spaces_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_spaces_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_spaces_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.delete_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.digit_to_end_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.digit_to_end_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.digit_to_end_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.digit_to_end_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.first_character_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.first_character_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.first_character_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.first_character_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.generate_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.generate_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.generate_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.generate_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.generate_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.generate_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_after_at_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_after_at_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_after_at_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_after_at_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_after_at_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_after_at_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_after_at_4_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_after_at_4_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_after_comma_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_after_comma_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_after_comma_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_after_comma_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_between_commas_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_between_commas_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_between_commas_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_between_commas_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_caps_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_caps_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_caps_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_caps_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_caps_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_caps_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_day_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_day_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_day_ordinal_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_day_ordinal_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_day_ordinal_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_day_ordinal_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_domain_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_domain_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_domain_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_domain_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_hour_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_hour_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_hour_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_hour_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_minutes_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_minutes_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_minutes_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_minutes_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_number_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_number_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_number_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_number_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_system_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_system_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_system_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_system_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_units_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_units_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_units_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_units_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_value_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_value_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_value_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_value_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_week_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_week_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_week_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.get_week_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.login_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.login_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.login_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.login_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_3_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_3_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_4_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_4_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_5_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_5_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_6_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_6_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_7_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_7_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_8_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_8_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_9_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.reduce_name_9_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_prefix_4_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_prefix_4_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_prefix_5_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_prefix_5_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_prefix_6_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_prefix_6_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_punctuation_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_punctuation_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_punctuation_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.set_punctuation_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.some_before_at_na_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.some_before_at_na_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.to_upper_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.to_upper_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.to_upper_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.4_examples.to_upper_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.add_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.add_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.add_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.add_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.add_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.add_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.add_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.add_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.add_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.add_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.add_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.add_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.after_symbol_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.after_symbol_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.after_symbol_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.after_symbol_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.append_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.append_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.append_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.append_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.append_time_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.append_time_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.append_time_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.append_time_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.between_symbols_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.between_symbols_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.between_symbols_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.between_symbols_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.brackets_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.brackets_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.brackets_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.brackets_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.change_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.change_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.change_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.change_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.change_format_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.change_format_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.change_format_4_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.change_format_4_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.change_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.change_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.change_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.change_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_10_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_10_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_3_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_3_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_4_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_4_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_5_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_5_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_6_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_6_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_7_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_7_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_8_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_8_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_9_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.convert_9_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.country_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.country_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.country_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.country_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.country_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.country_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.country_prefix_7_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.country_prefix_7_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.country_prefix_8_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.country_prefix_8_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.country_prefix_9_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.country_prefix_9_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_parentheses_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_parentheses_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_parentheses_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_parentheses_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_punctuation_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_punctuation_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_punctuation_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_punctuation_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_punctuation_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_punctuation_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_punctuation_4_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_punctuation_4_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_spaces_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_spaces_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_spaces_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_spaces_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.delete_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.digit_to_end_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.digit_to_end_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.digit_to_end_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.digit_to_end_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.first_character_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.first_character_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.first_character_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.first_character_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.generate_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.generate_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.generate_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.generate_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.generate_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.generate_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_after_at_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_after_at_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_after_at_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_after_at_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_after_at_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_after_at_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_after_at_4_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_after_at_4_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_after_comma_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_after_comma_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_after_comma_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_after_comma_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_between_commas_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_between_commas_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_between_commas_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_between_commas_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_caps_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_caps_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_caps_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_caps_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_caps_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_caps_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_day_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_day_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_day_ordinal_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_day_ordinal_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_day_ordinal_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_day_ordinal_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_domain_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_domain_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_domain_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_domain_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_hour_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_hour_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_hour_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_hour_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_minutes_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_minutes_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_minutes_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_minutes_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_number_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_number_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_number_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_number_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_system_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_system_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_system_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_system_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_units_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_units_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_units_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_units_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_value_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_value_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_value_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_value_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_week_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_week_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_week_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.get_week_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.login_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.login_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.login_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.login_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_3_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_3_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_4_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_4_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_5_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_5_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_6_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_6_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_7_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_7_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_8_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_8_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_9_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.reduce_name_9_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_prefix_4_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_prefix_4_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_prefix_5_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_prefix_5_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_prefix_6_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_prefix_6_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_punctuation_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_punctuation_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_punctuation_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.set_punctuation_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.some_before_at_na_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.some_before_at_na_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.to_upper_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.to_upper_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.to_upper_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples.to_upper_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.add_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.add_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.add_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.add_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.add_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.add_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.add_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.add_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.add_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.add_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.add_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.add_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.after_symbol_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.after_symbol_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.after_symbol_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.after_symbol_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.append_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.append_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.append_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.append_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.append_time_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.append_time_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.append_time_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.append_time_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.between_symbols_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.between_symbols_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.between_symbols_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.between_symbols_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.brackets_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.brackets_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.brackets_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.brackets_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.change_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.change_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.change_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.change_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.change_format_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.change_format_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.change_format_4_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.change_format_4_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.change_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.change_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.change_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.change_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_10_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_10_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_3_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_3_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_4_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_4_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_5_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_5_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_6_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_6_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_7_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_7_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_8_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_8_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_9_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.convert_9_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.country_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.country_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.country_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.country_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.country_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.country_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.country_prefix_7_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.country_prefix_7_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.country_prefix_8_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.country_prefix_8_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.country_prefix_9_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.country_prefix_9_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_parentheses_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_parentheses_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_parentheses_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_parentheses_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_punctuation_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_punctuation_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_punctuation_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_punctuation_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_punctuation_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_punctuation_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_punctuation_4_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_punctuation_4_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_spaces_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_spaces_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_spaces_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_spaces_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.delete_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.digit_to_end_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.digit_to_end_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.digit_to_end_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.digit_to_end_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.first_character_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.first_character_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.first_character_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.first_character_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.generate_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.generate_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.generate_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.generate_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.generate_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.generate_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_after_at_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_after_at_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_after_at_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_after_at_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_after_at_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_after_at_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_after_at_4_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_after_at_4_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_after_comma_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_after_comma_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_after_comma_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_after_comma_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_between_commas_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_between_commas_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_between_commas_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_between_commas_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_caps_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_caps_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_caps_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_caps_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_caps_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_caps_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_day_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_day_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_day_ordinal_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_day_ordinal_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_day_ordinal_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_day_ordinal_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_domain_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_domain_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_domain_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_domain_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_hour_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_hour_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_hour_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_hour_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_minutes_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_minutes_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_minutes_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_minutes_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_number_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_number_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_number_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_number_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_system_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_system_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_system_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_system_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_units_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_units_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_units_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_units_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_value_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_value_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_value_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_value_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_week_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_week_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_week_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.get_week_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.login_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.login_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.login_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.login_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_3_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_3_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_4_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_4_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_5_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_5_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_6_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_6_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_7_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_7_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_8_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_8_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_9_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.reduce_name_9_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_prefix_4_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_prefix_4_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_prefix_5_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_prefix_5_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_prefix_6_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_prefix_6_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_punctuation_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_punctuation_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_punctuation_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.set_punctuation_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.some_before_at_na_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.some_before_at_na_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.to_upper_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.to_upper_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.to_upper_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.4_examples.to_upper_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.add_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.add_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.add_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.add_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.add_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.add_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.add_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.add_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.add_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.add_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.add_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.add_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.after_symbol_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.after_symbol_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.after_symbol_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.after_symbol_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.append_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.append_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.append_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.append_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.append_time_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.append_time_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.append_time_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.append_time_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.between_symbols_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.between_symbols_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.between_symbols_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.between_symbols_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.brackets_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.brackets_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.brackets_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.brackets_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.change_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.change_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.change_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.change_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.change_format_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.change_format_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.change_format_4_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.change_format_4_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.change_punctuation_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.change_punctuation_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.change_punctuation_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.change_punctuation_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_10_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_10_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_3_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_3_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_3_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_3_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_4_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_4_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_4_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_4_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_5_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_5_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_6_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_6_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_7_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_7_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_8_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_8_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_9_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.convert_9_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.country_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.country_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.country_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.country_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.country_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.country_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.country_prefix_7_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.country_prefix_7_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.country_prefix_8_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.country_prefix_8_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.country_prefix_9_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.country_prefix_9_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_parentheses_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_parentheses_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_parentheses_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_parentheses_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_punctuation_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_punctuation_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_punctuation_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_punctuation_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_punctuation_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_punctuation_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_punctuation_4_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_punctuation_4_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_spaces_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_spaces_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_spaces_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_spaces_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.delete_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.digit_to_end_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.digit_to_end_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.digit_to_end_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.digit_to_end_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.first_character_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.first_character_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.first_character_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.first_character_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.generate_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.generate_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.generate_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.generate_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.generate_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.generate_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_after_at_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_after_at_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_after_at_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_after_at_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_after_at_3_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_after_at_3_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_after_at_4_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_after_at_4_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_after_comma_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_after_comma_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_after_comma_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_after_comma_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_between_commas_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_between_commas_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_between_commas_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_between_commas_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_caps_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_caps_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_caps_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_caps_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_caps_3_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_caps_3_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_day_3_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_day_3_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_day_ordinal_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_day_ordinal_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_day_ordinal_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_day_ordinal_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_domain_1_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_domain_1_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_domain_2_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_domain_2_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_hour_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_hour_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_hour_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_hour_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_minutes_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_minutes_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_minutes_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_minutes_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_number_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_number_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_number_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_number_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_system_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_system_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_system_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_system_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_time_1_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_time_1_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_time_2_times_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_time_2_times_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_title_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_title_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_title_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_title_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_units_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_units_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_units_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_units_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_value_1_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_value_1_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_value_2_units_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_value_2_units_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_week_day_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_week_day_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_week_day_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.get_week_day_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.login_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.login_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.login_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.login_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_month_name_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_month_name_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_month_name_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_month_name_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_1_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_1_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_2_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_2_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_3_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_3_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_4_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_4_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_5_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_5_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_6_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_6_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_7_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_7_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_8_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_8_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_9_names_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.reduce_name_9_names_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_format_1_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_format_1_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_format_2_dates_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_format_2_dates_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_prefix_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_prefix_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_prefix_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_prefix_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_prefix_3_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_prefix_3_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_prefix_4_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_prefix_4_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_prefix_5_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_prefix_5_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_prefix_6_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_prefix_6_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_punctuation_1_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_punctuation_1_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_punctuation_2_phones_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.set_punctuation_2_phones_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.some_before_at_na_emails_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.some_before_at_na_emails_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.to_upper_1_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.to_upper_1_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.to_upper_2_freetext_1`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples.to_upper_2_freetext_2`<br>`bigbench:mult_data_wrangling.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>add_punctuation_1_dates_1<br>add_punctuation_1_dates_2<br>add_punctuation_2_dates_1<br>add_punctuation_2_dates_2<br>add_time_1_times_1<br>add_time_1_times_2<br>add_time_2_times_1<br>add_time_2_times_2<br>add_title_1_names_1<br>add_title_1_names_2<br>add_title_2_names_1<br>add_title_2_names_2<br>after_symbol_1_freetext_1<br>after_symbol_1_freetext_2<br>after_symbol_2_freetext_1<br>after_symbol_2_freetext_2<br>append_time_1_times_1<br>append_time_1_times_2<br>append_time_2_times_1<br>append_time_2_times_2<br>append_time_3_times_1<br>append_time_3_times_2<br>append_time_4_times_1<br>append_time_4_times_2<br>between_symbols_1_freetext_1<br>between_symbols_1_freetext_2<br>between_symbols_2_freetext_1<br>between_symbols_2_freetext_2<br>brackets_1_freetext_1<br>brackets_1_freetext_2<br>brackets_2_freetext_1<br>brackets_2_freetext_2<br>change_format_1_dates_1<br>change_format_1_dates_2<br>change_format_2_dates_1<br>change_format_2_dates_2<br>change_format_3_dates_1<br>change_format_3_dates_2<br>change_format_4_dates_1<br>change_format_4_dates_2<br>change_punctuation_1_dates_1<br>change_punctuation_1_dates_2<br>change_punctuation_2_dates_1<br>change_punctuation_2_dates_2<br>convert_10_times_1<br>convert_10_times_2<br>convert_1_times_1<br>convert_1_times_2<br>convert_1_units_1<br>convert_1_units_2<br>convert_2_times_1<br>convert_2_times_2<br>convert_2_units_1<br>convert_2_units_2<br>convert_3_times_1<br>convert_3_times_2<br>convert_3_units_1<br>convert_3_units_2<br>convert_4_times_1<br>convert_4_times_2<br>convert_4_units_1<br>convert_4_units_2<br>convert_5_times_1<br>convert_5_times_2<br>convert_6_times_1<br>convert_6_times_2<br>convert_7_times_1<br>convert_7_times_2<br>convert_8_times_1<br>convert_8_times_2<br>convert_9_times_1<br>convert_9_times_2<br>country_prefix_1_phones_1<br>country_prefix_1_phones_2<br>country_prefix_2_phones_1<br>country_prefix_2_phones_2<br>country_prefix_3_phones_1<br>country_prefix_3_phones_2<br>country_prefix_7_phones_1<br>country_prefix_7_phones_2<br>country_prefix_8_phones_1<br>country_prefix_8_phones_2<br>country_prefix_9_phones_1<br>country_prefix_9_phones_2<br>delete_parentheses_1_phones_1<br>delete_parentheses_1_phones_2<br>delete_parentheses_2_phones_1<br>delete_parentheses_2_phones_2<br>delete_punctuation_1_freetext_1<br>delete_punctuation_1_freetext_2<br>delete_punctuation_2_freetext_1<br>delete_punctuation_2_freetext_2<br>delete_punctuation_3_freetext_1<br>delete_punctuation_3_freetext_2<br>delete_punctuation_4_freetext_1<br>delete_punctuation_4_freetext_2<br>delete_spaces_1_freetext_1<br>delete_spaces_1_freetext_2<br>delete_spaces_2_freetext_1<br>delete_spaces_2_freetext_2<br>delete_time_1_times_1<br>delete_time_1_times_2<br>delete_time_2_times_1<br>delete_time_2_times_2<br>digit_to_end_1_freetext_1<br>digit_to_end_1_freetext_2<br>digit_to_end_2_freetext_1<br>digit_to_end_2_freetext_2<br>first_character_1_freetext_1<br>first_character_1_freetext_2<br>first_character_2_freetext_1<br>first_character_2_freetext_2<br>generate_1_emails_1<br>generate_1_emails_2<br>generate_2_emails_1<br>generate_2_emails_2<br>generate_3_emails_1<br>generate_3_emails_2<br>get_after_at_1_emails_1<br>get_after_at_1_emails_2<br>get_after_at_2_emails_1<br>get_after_at_2_emails_2<br>get_after_at_3_emails_1<br>get_after_at_3_emails_2<br>get_after_at_4_emails_1<br>get_after_at_4_emails_2<br>get_after_comma_1_freetext_1<br>get_after_comma_1_freetext_2<br>get_after_comma_2_freetext_1<br>get_after_comma_2_freetext_2<br>get_between_commas_1_freetext_1<br>get_between_commas_1_freetext_2<br>get_between_commas_2_freetext_1<br>get_between_commas_2_freetext_2<br>get_caps_1_freetext_1<br>get_caps_1_freetext_2<br>get_caps_2_freetext_1<br>get_caps_2_freetext_2<br>get_caps_3_freetext_1<br>get_caps_3_freetext_2<br>get_day_1_dates_1<br>get_day_1_dates_2<br>get_day_2_dates_1<br>get_day_2_dates_2<br>get_day_3_dates_1<br>get_day_3_dates_2<br>get_day_ordinal_1_dates_1<br>get_day_ordinal_1_dates_2<br>get_day_ordinal_2_dates_1<br>get_day_ordinal_2_dates_2<br>get_domain_1_emails_1<br>get_domain_1_emails_2<br>get_domain_2_emails_1<br>get_domain_2_emails_2<br>get_hour_1_times_1<br>get_hour_1_times_2<br>get_hour_2_times_1<br>get_hour_2_times_2<br>get_minutes_1_times_1<br>get_minutes_1_times_2<br>get_minutes_2_times_1<br>get_minutes_2_times_2<br>get_month_name_1_dates_1<br>get_month_name_1_dates_2<br>get_month_name_2_dates_1<br>get_month_name_2_dates_2<br>get_number_1_phones_1<br>get_number_1_phones_2<br>get_number_2_phones_1<br>get_number_2_phones_2<br>get_system_1_units_1<br>get_system_1_units_2<br>get_system_2_units_1<br>get_system_2_units_2<br>get_time_1_times_1<br>get_time_1_times_2<br>get_time_2_times_1<br>get_time_2_times_2<br>get_title_1_names_1<br>get_title_1_names_2<br>get_title_2_names_1<br>get_title_2_names_2<br>get_units_1_units_1<br>get_units_1_units_2<br>get_units_2_units_1<br>get_units_2_units_2<br>get_value_1_units_1<br>get_value_1_units_2<br>get_value_2_units_1<br>get_value_2_units_2<br>get_week_day_1_dates_1<br>get_week_day_1_dates_2<br>get_week_day_2_dates_1<br>get_week_day_2_dates_2<br>login_1_names_1<br>login_1_names_2<br>login_2_names_1<br>login_2_names_2<br>reduce_month_name_1_dates_1<br>reduce_month_name_1_dates_2<br>reduce_month_name_2_dates_1<br>reduce_month_name_2_dates_2<br>reduce_name_1_names_1<br>reduce_name_1_names_2<br>reduce_name_2_names_1<br>reduce_name_2_names_2<br>reduce_name_3_names_1<br>reduce_name_3_names_2<br>reduce_name_4_names_1<br>reduce_name_4_names_2<br>reduce_name_5_names_1<br>reduce_name_5_names_2<br>reduce_name_6_names_1<br>reduce_name_6_names_2<br>reduce_name_7_names_1<br>reduce_name_7_names_2<br>reduce_name_8_names_1<br>reduce_name_8_names_2<br>reduce_name_9_names_1<br>reduce_name_9_names_2<br>set_format_1_dates_1<br>set_format_1_dates_2<br>set_format_2_dates_1<br>set_format_2_dates_2<br>set_prefix_1_phones_1<br>set_prefix_1_phones_2<br>set_prefix_2_phones_1<br>set_prefix_2_phones_2<br>set_prefix_3_phones_1<br>set_prefix_3_phones_2<br>set_prefix_4_phones_1<br>set_prefix_4_phones_2<br>set_prefix_5_phones_1<br>set_prefix_5_phones_2<br>set_prefix_6_phones_1<br>set_prefix_6_phones_2<br>set_punctuation_1_phones_1<br>set_punctuation_1_phones_2<br>set_punctuation_2_phones_1<br>set_punctuation_2_phones_2<br>some_before_at_na_emails_1<br>some_before_at_na_emails_2<br>to_upper_1_freetext_1<br>to_upper_1_freetext_2<br>to_upper_2_freetext_1<br>to_upper_2_freetext_2</details> | <details><summary>Description</summary><br>Perform multiple-domain data wrangling tasks</details> | Generative, Scoring
[multiemo](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/multiemo/) | <details><summary>Tasks</summary><br>`bigbench:multiemo.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:multiemo.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:multiemo.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:multiemo.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:multiemo.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:multiemo.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:multiemo.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:multiemo.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>all_sentence_de<br>all_sentence_en<br>all_sentence_es<br>all_sentence_fr<br>all_sentence_it<br>all_sentence_ja<br>all_sentence_nl<br>all_sentence_pl<br>all_sentence_pt<br>all_sentence_ru<br>all_sentence_zh<br>all_text_de<br>all_text_en<br>all_text_es<br>all_text_fr<br>all_text_it<br>all_text_ja<br>all_text_nl<br>all_text_pl<br>all_text_pt<br>all_text_ru<br>all_text_zh<br>hotels_sentence_de<br>hotels_sentence_en<br>hotels_sentence_es<br>hotels_sentence_fr<br>hotels_sentence_it<br>hotels_sentence_ja<br>hotels_sentence_nl<br>hotels_sentence_pl<br>hotels_sentence_pt<br>hotels_sentence_ru<br>hotels_sentence_zh<br>hotels_text_de<br>hotels_text_en<br>hotels_text_es<br>hotels_text_fr<br>hotels_text_it<br>hotels_text_ja<br>hotels_text_nl<br>hotels_text_pl<br>hotels_text_pt<br>hotels_text_ru<br>hotels_text_zh<br>medicine_sentence_de<br>medicine_sentence_en<br>medicine_sentence_es<br>medicine_sentence_fr<br>medicine_sentence_it<br>medicine_sentence_ja<br>medicine_sentence_nl<br>medicine_sentence_pl<br>medicine_sentence_pt<br>medicine_sentence_ru<br>medicine_sentence_zh<br>medicine_text_de<br>medicine_text_en<br>medicine_text_es<br>medicine_text_fr<br>medicine_text_it<br>medicine_text_ja<br>medicine_text_nl<br>medicine_text_pl<br>medicine_text_pt<br>medicine_text_ru<br>medicine_text_zh<br>products_sentence_de<br>products_sentence_en<br>products_sentence_es<br>products_sentence_fr<br>products_sentence_it<br>products_sentence_ja<br>products_sentence_nl<br>products_sentence_pl<br>products_sentence_pt<br>products_sentence_ru<br>products_sentence_zh<br>products_text_de<br>products_text_en<br>products_text_es<br>products_text_fr<br>products_text_it<br>products_text_ja<br>products_text_nl<br>products_text_pl<br>products_text_pt<br>products_text_ru<br>products_text_zh<br>reviews_sentence_de<br>reviews_sentence_en<br>reviews_sentence_es<br>reviews_sentence_fr<br>reviews_sentence_it<br>reviews_sentence_ja<br>reviews_sentence_nl<br>reviews_sentence_pl<br>reviews_sentence_pt<br>reviews_sentence_ru<br>reviews_sentence_zh<br>reviews_text_de<br>reviews_text_en<br>reviews_text_es<br>reviews_text_fr<br>reviews_text_it<br>reviews_text_ja<br>reviews_text_nl<br>reviews_text_pl<br>reviews_text_pt<br>reviews_text_ru<br>reviews_text_zh</details> | <details><summary>Description</summary><br>Analyze the sentiment of customer reviews</details> | Multiple-Choice
[natural_instructions](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/natural_instructions/) | <details><summary>Tasks</summary><br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask001_quoref_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask002_quoref_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask003_mctaco_question_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask004_mctaco_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask005_mctaco_wrong_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask006_mctaco_question_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask007_mctaco_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask008_mctaco_wrong_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask009_mctaco_question_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask010_mctaco_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask011_mctaco_wrong_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask012_mctaco_question_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask013_mctaco_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask014_mctaco_wrong_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask015_mctaco_question_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask016_mctaco_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask017_mctaco_wrong_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask018_mctaco_temporal_reasoning_presence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask019_mctaco_temporal_reasoning_category`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask020_mctaco_span_based_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask021_mctaco_grammatical_logical`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask022_cosmosqa_passage_inappropriate_binary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask023_cosmosqa_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask024_cosmosqa_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask025_cosmosqa_incorrect_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask026_drop_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask027_drop_answer_type_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask028_drop_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask029_winogrande_full_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask030_winogrande_full_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask031_winogrande_question_generation_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask032_winogrande_question_generation_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask033_winogrande_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask034_winogrande_question_modification_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask035_winogrande_question_modification_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask036_qasc_topic_word_to_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask037_qasc_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask038_qasc_combined_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask039_qasc_find_overlapping_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask040_qasc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask041_qasc_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask042_qasc_incorrect_option_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask043_essential_terms_answering_incomplete_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask044_essential_terms_identifying_essential_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask045_miscellaneous_sentence_paraphrasing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask046_miscellaenous_question_typing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask047_misc_answering_science_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask048_multirc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask049_multirc_questions_needed_to_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask050_multirc_answerability`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask051_multirc_correct_answer_single_sentence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask052_multirc_identify_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask053_multirc_correct_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask054_multirc_write_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask055_multirc_write_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask056_multirc_classify_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask057_multirc_classify_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask058_multirc_question_answering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask059_ropes_story_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask060_ropes_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.16_examples.subtask061_ropes_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask001_quoref_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask002_quoref_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask003_mctaco_question_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask004_mctaco_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask005_mctaco_wrong_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask006_mctaco_question_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask007_mctaco_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask008_mctaco_wrong_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask009_mctaco_question_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask010_mctaco_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask011_mctaco_wrong_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask012_mctaco_question_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask013_mctaco_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask014_mctaco_wrong_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask015_mctaco_question_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask016_mctaco_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask017_mctaco_wrong_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask018_mctaco_temporal_reasoning_presence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask019_mctaco_temporal_reasoning_category`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask020_mctaco_span_based_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask021_mctaco_grammatical_logical`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask022_cosmosqa_passage_inappropriate_binary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask023_cosmosqa_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask024_cosmosqa_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask025_cosmosqa_incorrect_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask026_drop_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask027_drop_answer_type_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask028_drop_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask029_winogrande_full_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask030_winogrande_full_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask031_winogrande_question_generation_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask032_winogrande_question_generation_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask033_winogrande_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask034_winogrande_question_modification_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask035_winogrande_question_modification_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask036_qasc_topic_word_to_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask037_qasc_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask038_qasc_combined_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask039_qasc_find_overlapping_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask040_qasc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask041_qasc_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask042_qasc_incorrect_option_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask043_essential_terms_answering_incomplete_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask044_essential_terms_identifying_essential_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask045_miscellaneous_sentence_paraphrasing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask046_miscellaenous_question_typing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask047_misc_answering_science_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask048_multirc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask049_multirc_questions_needed_to_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask050_multirc_answerability`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask051_multirc_correct_answer_single_sentence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask052_multirc_identify_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask053_multirc_correct_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask054_multirc_write_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask055_multirc_write_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask056_multirc_classify_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask057_multirc_classify_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask058_multirc_question_answering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask059_ropes_story_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask060_ropes_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples.subtask061_ropes_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask001_quoref_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask002_quoref_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask003_mctaco_question_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask004_mctaco_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask005_mctaco_wrong_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask006_mctaco_question_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask007_mctaco_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask008_mctaco_wrong_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask009_mctaco_question_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask010_mctaco_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask011_mctaco_wrong_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask012_mctaco_question_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask013_mctaco_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask014_mctaco_wrong_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask015_mctaco_question_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask016_mctaco_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask017_mctaco_wrong_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask018_mctaco_temporal_reasoning_presence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask019_mctaco_temporal_reasoning_category`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask020_mctaco_span_based_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask021_mctaco_grammatical_logical`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask022_cosmosqa_passage_inappropriate_binary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask023_cosmosqa_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask024_cosmosqa_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask025_cosmosqa_incorrect_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask026_drop_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask027_drop_answer_type_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask028_drop_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask029_winogrande_full_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask030_winogrande_full_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask031_winogrande_question_generation_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask032_winogrande_question_generation_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask033_winogrande_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask034_winogrande_question_modification_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask035_winogrande_question_modification_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask036_qasc_topic_word_to_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask037_qasc_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask038_qasc_combined_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask039_qasc_find_overlapping_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask040_qasc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask041_qasc_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask042_qasc_incorrect_option_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask043_essential_terms_answering_incomplete_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask044_essential_terms_identifying_essential_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask045_miscellaneous_sentence_paraphrasing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask046_miscellaenous_question_typing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask047_misc_answering_science_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask048_multirc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask049_multirc_questions_needed_to_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask050_multirc_answerability`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask051_multirc_correct_answer_single_sentence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask052_multirc_identify_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask053_multirc_correct_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask054_multirc_write_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask055_multirc_write_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask056_multirc_classify_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask057_multirc_classify_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask058_multirc_question_answering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask059_ropes_story_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask060_ropes_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.16_examples.subtask061_ropes_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask001_quoref_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask002_quoref_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask003_mctaco_question_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask004_mctaco_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask005_mctaco_wrong_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask006_mctaco_question_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask007_mctaco_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask008_mctaco_wrong_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask009_mctaco_question_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask010_mctaco_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask011_mctaco_wrong_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask012_mctaco_question_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask013_mctaco_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask014_mctaco_wrong_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask015_mctaco_question_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask016_mctaco_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask017_mctaco_wrong_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask018_mctaco_temporal_reasoning_presence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask019_mctaco_temporal_reasoning_category`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask020_mctaco_span_based_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask021_mctaco_grammatical_logical`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask022_cosmosqa_passage_inappropriate_binary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask023_cosmosqa_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask024_cosmosqa_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask025_cosmosqa_incorrect_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask026_drop_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask027_drop_answer_type_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask028_drop_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask029_winogrande_full_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask030_winogrande_full_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask031_winogrande_question_generation_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask032_winogrande_question_generation_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask033_winogrande_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask034_winogrande_question_modification_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask035_winogrande_question_modification_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask036_qasc_topic_word_to_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask037_qasc_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask038_qasc_combined_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask039_qasc_find_overlapping_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask040_qasc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask041_qasc_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask042_qasc_incorrect_option_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask043_essential_terms_answering_incomplete_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask044_essential_terms_identifying_essential_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask045_miscellaneous_sentence_paraphrasing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask046_miscellaenous_question_typing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask047_misc_answering_science_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask048_multirc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask049_multirc_questions_needed_to_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask050_multirc_answerability`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask051_multirc_correct_answer_single_sentence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask052_multirc_identify_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask053_multirc_correct_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask054_multirc_write_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask055_multirc_write_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask056_multirc_classify_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask057_multirc_classify_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask058_multirc_question_answering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask059_ropes_story_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask060_ropes_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples.subtask061_ropes_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask001_quoref_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask002_quoref_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask003_mctaco_question_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask004_mctaco_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask005_mctaco_wrong_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask006_mctaco_question_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask007_mctaco_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask008_mctaco_wrong_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask009_mctaco_question_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask010_mctaco_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask011_mctaco_wrong_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask012_mctaco_question_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask013_mctaco_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask014_mctaco_wrong_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask015_mctaco_question_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask016_mctaco_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask017_mctaco_wrong_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask018_mctaco_temporal_reasoning_presence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask019_mctaco_temporal_reasoning_category`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask020_mctaco_span_based_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask021_mctaco_grammatical_logical`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask022_cosmosqa_passage_inappropriate_binary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask023_cosmosqa_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask024_cosmosqa_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask025_cosmosqa_incorrect_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask026_drop_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask027_drop_answer_type_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask028_drop_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask029_winogrande_full_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask030_winogrande_full_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask031_winogrande_question_generation_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask032_winogrande_question_generation_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask033_winogrande_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask034_winogrande_question_modification_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask035_winogrande_question_modification_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask036_qasc_topic_word_to_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask037_qasc_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask038_qasc_combined_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask039_qasc_find_overlapping_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask040_qasc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask041_qasc_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask042_qasc_incorrect_option_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask043_essential_terms_answering_incomplete_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask044_essential_terms_identifying_essential_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask045_miscellaneous_sentence_paraphrasing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask046_miscellaenous_question_typing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask047_misc_answering_science_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask048_multirc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask049_multirc_questions_needed_to_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask050_multirc_answerability`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask051_multirc_correct_answer_single_sentence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask052_multirc_identify_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask053_multirc_correct_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask054_multirc_write_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask055_multirc_write_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask056_multirc_classify_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask057_multirc_classify_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask058_multirc_question_answering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask059_ropes_story_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask060_ropes_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.16_examples.subtask061_ropes_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask001_quoref_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask002_quoref_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask003_mctaco_question_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask004_mctaco_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask005_mctaco_wrong_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask006_mctaco_question_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask007_mctaco_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask008_mctaco_wrong_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask009_mctaco_question_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask010_mctaco_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask011_mctaco_wrong_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask012_mctaco_question_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask013_mctaco_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask014_mctaco_wrong_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask015_mctaco_question_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask016_mctaco_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask017_mctaco_wrong_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask018_mctaco_temporal_reasoning_presence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask019_mctaco_temporal_reasoning_category`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask020_mctaco_span_based_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask021_mctaco_grammatical_logical`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask022_cosmosqa_passage_inappropriate_binary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask023_cosmosqa_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask024_cosmosqa_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask025_cosmosqa_incorrect_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask026_drop_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask027_drop_answer_type_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask028_drop_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask029_winogrande_full_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask030_winogrande_full_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask031_winogrande_question_generation_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask032_winogrande_question_generation_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask033_winogrande_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask034_winogrande_question_modification_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask035_winogrande_question_modification_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask036_qasc_topic_word_to_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask037_qasc_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask038_qasc_combined_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask039_qasc_find_overlapping_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask040_qasc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask041_qasc_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask042_qasc_incorrect_option_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask043_essential_terms_answering_incomplete_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask044_essential_terms_identifying_essential_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask045_miscellaneous_sentence_paraphrasing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask046_miscellaenous_question_typing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask047_misc_answering_science_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask048_multirc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask049_multirc_questions_needed_to_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask050_multirc_answerability`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask051_multirc_correct_answer_single_sentence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask052_multirc_identify_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask053_multirc_correct_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask054_multirc_write_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask055_multirc_write_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask056_multirc_classify_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask057_multirc_classify_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask058_multirc_question_answering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask059_ropes_story_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask060_ropes_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples.subtask061_ropes_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask001_quoref_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask002_quoref_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask003_mctaco_question_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask004_mctaco_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask005_mctaco_wrong_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask006_mctaco_question_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask007_mctaco_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask008_mctaco_wrong_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask009_mctaco_question_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask010_mctaco_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask011_mctaco_wrong_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask012_mctaco_question_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask013_mctaco_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask014_mctaco_wrong_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask015_mctaco_question_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask016_mctaco_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask017_mctaco_wrong_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask018_mctaco_temporal_reasoning_presence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask019_mctaco_temporal_reasoning_category`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask020_mctaco_span_based_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask021_mctaco_grammatical_logical`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask022_cosmosqa_passage_inappropriate_binary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask023_cosmosqa_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask024_cosmosqa_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask025_cosmosqa_incorrect_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask026_drop_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask027_drop_answer_type_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask028_drop_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask029_winogrande_full_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask030_winogrande_full_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask031_winogrande_question_generation_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask032_winogrande_question_generation_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask033_winogrande_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask034_winogrande_question_modification_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask035_winogrande_question_modification_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask036_qasc_topic_word_to_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask037_qasc_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask038_qasc_combined_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask039_qasc_find_overlapping_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask040_qasc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask041_qasc_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask042_qasc_incorrect_option_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask043_essential_terms_answering_incomplete_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask044_essential_terms_identifying_essential_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask045_miscellaneous_sentence_paraphrasing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask046_miscellaenous_question_typing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask047_misc_answering_science_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask048_multirc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask049_multirc_questions_needed_to_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask050_multirc_answerability`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask051_multirc_correct_answer_single_sentence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask052_multirc_identify_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask053_multirc_correct_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask054_multirc_write_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask055_multirc_write_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask056_multirc_classify_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask057_multirc_classify_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask058_multirc_question_answering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask059_ropes_story_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask060_ropes_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.16_examples.subtask061_ropes_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask001_quoref_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask002_quoref_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask003_mctaco_question_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask004_mctaco_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask005_mctaco_wrong_answer_generation_event_duration`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask006_mctaco_question_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask007_mctaco_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask008_mctaco_wrong_answer_generation_transient_stationary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask009_mctaco_question_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask010_mctaco_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask011_mctaco_wrong_answer_generation_event_ordering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask012_mctaco_question_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask013_mctaco_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask014_mctaco_wrong_answer_generation_absolute_timepoint`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask015_mctaco_question_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask016_mctaco_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask017_mctaco_wrong_answer_generation_frequency`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask018_mctaco_temporal_reasoning_presence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask019_mctaco_temporal_reasoning_category`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask020_mctaco_span_based_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask021_mctaco_grammatical_logical`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask022_cosmosqa_passage_inappropriate_binary`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask023_cosmosqa_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask024_cosmosqa_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask025_cosmosqa_incorrect_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask026_drop_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask027_drop_answer_type_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask028_drop_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask029_winogrande_full_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask030_winogrande_full_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask031_winogrande_question_generation_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask032_winogrande_question_generation_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask033_winogrande_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask034_winogrande_question_modification_object`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask035_winogrande_question_modification_person`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask036_qasc_topic_word_to_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask037_qasc_generate_related_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask038_qasc_combined_fact`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask039_qasc_find_overlapping_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask040_qasc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask041_qasc_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask042_qasc_incorrect_option_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask043_essential_terms_answering_incomplete_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask044_essential_terms_identifying_essential_words`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask045_miscellaneous_sentence_paraphrasing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask046_miscellaenous_question_typing`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask047_misc_answering_science_questions`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask048_multirc_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask049_multirc_questions_needed_to_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask050_multirc_answerability`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask051_multirc_correct_answer_single_sentence`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask052_multirc_identify_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask053_multirc_correct_bad_question`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask054_multirc_write_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask055_multirc_write_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask056_multirc_classify_correct_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask057_multirc_classify_incorrect_answer`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask058_multirc_question_answering`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask059_ropes_story_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask060_ropes_question_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples.subtask061_ropes_answer_generation`<br>`bigbench:natural_instructions.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>subtask001_quoref_question_generation<br>subtask002_quoref_answer_generation<br>subtask003_mctaco_question_generation_event_duration<br>subtask004_mctaco_answer_generation_event_duration<br>subtask005_mctaco_wrong_answer_generation_event_duration<br>subtask006_mctaco_question_generation_transient_stationary<br>subtask007_mctaco_answer_generation_transient_stationary<br>subtask008_mctaco_wrong_answer_generation_transient_stationary<br>subtask009_mctaco_question_generation_event_ordering<br>subtask010_mctaco_answer_generation_event_ordering<br>subtask011_mctaco_wrong_answer_generation_event_ordering<br>subtask012_mctaco_question_generation_absolute_timepoint<br>subtask013_mctaco_answer_generation_absolute_timepoint<br>subtask014_mctaco_wrong_answer_generation_absolute_timepoint<br>subtask015_mctaco_question_generation_frequency<br>subtask016_mctaco_answer_generation_frequency<br>subtask017_mctaco_wrong_answer_generation_frequency<br>subtask018_mctaco_temporal_reasoning_presence<br>subtask019_mctaco_temporal_reasoning_category<br>subtask020_mctaco_span_based_question<br>subtask021_mctaco_grammatical_logical<br>subtask022_cosmosqa_passage_inappropriate_binary<br>subtask023_cosmosqa_question_generation<br>subtask024_cosmosqa_answer_generation<br>subtask025_cosmosqa_incorrect_answer_generation<br>subtask026_drop_question_generation<br>subtask027_drop_answer_type_generation<br>subtask028_drop_answer_generation<br>subtask029_winogrande_full_object<br>subtask030_winogrande_full_person<br>subtask031_winogrande_question_generation_object<br>subtask032_winogrande_question_generation_person<br>subtask033_winogrande_answer_generation<br>subtask034_winogrande_question_modification_object<br>subtask035_winogrande_question_modification_person<br>subtask036_qasc_topic_word_to_generate_related_fact<br>subtask037_qasc_generate_related_fact<br>subtask038_qasc_combined_fact<br>subtask039_qasc_find_overlapping_words<br>subtask040_qasc_question_generation<br>subtask041_qasc_answer_generation<br>subtask042_qasc_incorrect_option_generation<br>subtask043_essential_terms_answering_incomplete_questions<br>subtask044_essential_terms_identifying_essential_words<br>subtask045_miscellaneous_sentence_paraphrasing<br>subtask046_miscellaenous_question_typing<br>subtask047_misc_answering_science_questions<br>subtask048_multirc_question_generation<br>subtask049_multirc_questions_needed_to_answer<br>subtask050_multirc_answerability<br>subtask051_multirc_correct_answer_single_sentence<br>subtask052_multirc_identify_bad_question<br>subtask053_multirc_correct_bad_question<br>subtask054_multirc_write_correct_answer<br>subtask055_multirc_write_incorrect_answer<br>subtask056_multirc_classify_correct_answer<br>subtask057_multirc_classify_incorrect_answer<br>subtask058_multirc_question_answering<br>subtask059_ropes_story_generation<br>subtask060_ropes_question_generation<br>subtask061_ropes_answer_generation</details> | <details><summary>Description</summary><br>This dataset consists of 61 distinct tasks and the crowdsourcing instructions that were used to crowdsource them</details> | Generative, Scoring
[periodic_elements](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/periodic_elements/) | <details><summary>Tasks</summary><br>`bigbench:periodic_elements.mix.t5_default_vocab.0_shot.118_examples.subtask_0`<br>`bigbench:periodic_elements.mix.t5_default_vocab.0_shot.512_examples.subtask_1`<br>`bigbench:periodic_elements.mix.t5_default_vocab.0_shot.654_examples`<br>`bigbench:periodic_elements.mix.t5_default_vocab.0_shot.all_examples.subtask_0`<br>`bigbench:periodic_elements.mix.t5_default_vocab.0_shot.all_examples.subtask_1`<br>`bigbench:periodic_elements.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:periodic_elements.mix.t5_default_vocab.1_shot.118_examples.subtask_0`<br>`bigbench:periodic_elements.mix.t5_default_vocab.1_shot.512_examples.subtask_1`<br>`bigbench:periodic_elements.mix.t5_default_vocab.1_shot.654_examples`<br>`bigbench:periodic_elements.mix.t5_default_vocab.1_shot.all_examples.subtask_0`<br>`bigbench:periodic_elements.mix.t5_default_vocab.1_shot.all_examples.subtask_1`<br>`bigbench:periodic_elements.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:periodic_elements.mix.t5_default_vocab.2_shot.118_examples.subtask_0`<br>`bigbench:periodic_elements.mix.t5_default_vocab.2_shot.512_examples.subtask_1`<br>`bigbench:periodic_elements.mix.t5_default_vocab.2_shot.654_examples`<br>`bigbench:periodic_elements.mix.t5_default_vocab.2_shot.all_examples.subtask_0`<br>`bigbench:periodic_elements.mix.t5_default_vocab.2_shot.all_examples.subtask_1`<br>`bigbench:periodic_elements.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:periodic_elements.mix.t5_default_vocab.3_shot.118_examples.subtask_0`<br>`bigbench:periodic_elements.mix.t5_default_vocab.3_shot.512_examples.subtask_1`<br>`bigbench:periodic_elements.mix.t5_default_vocab.3_shot.654_examples`<br>`bigbench:periodic_elements.mix.t5_default_vocab.3_shot.all_examples.subtask_0`<br>`bigbench:periodic_elements.mix.t5_default_vocab.3_shot.all_examples.subtask_1`<br>`bigbench:periodic_elements.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>subtask_0<br>subtask_1</details> | <details><summary>Description</summary><br>Predict names of elements from the periodic table, given indirect descriptions of the element's place on the table</details> | Generative, Multiple-Choice, Scoring
[real_or_fake_text](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/real_or_fake_text/) | <details><summary>Tasks</summary><br>`bigbench:real_or_fake_text.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:real_or_fake_text.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:real_or_fake_text.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:real_or_fake_text.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:real_or_fake_text.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:real_or_fake_text.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:real_or_fake_text.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:real_or_fake_text.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>easy<br>gpt2<br>gpt2_xl</details> | <details><summary>Description</summary><br>Determine the sentence at which a document transitions from human written to machine generated</details> | Multiple-Choice
[simp_turing_concept](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/simp_turing_concept/) | <details><summary>Tasks</summary><br>`bigbench:simp_turing_concept.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.0_shot.341_examples.additional_set_1`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.0_shot.341_examples.additional_set_2`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.0_shot.341_examples.witness_set`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.0_shot.all_examples.additional_set_1`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.0_shot.all_examples.additional_set_2`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.0_shot.all_examples.witness_set`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.1_shot.341_examples.additional_set_1`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.1_shot.341_examples.additional_set_2`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.1_shot.341_examples.witness_set`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.1_shot.all_examples.additional_set_1`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.1_shot.all_examples.additional_set_2`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.1_shot.all_examples.witness_set`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.2_shot.341_examples.additional_set_1`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.2_shot.341_examples.additional_set_2`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.2_shot.341_examples.witness_set`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.2_shot.all_examples.additional_set_1`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.2_shot.all_examples.additional_set_2`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.2_shot.all_examples.witness_set`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.3_shot.341_examples.additional_set_1`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.3_shot.341_examples.additional_set_2`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.3_shot.341_examples.witness_set`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.3_shot.all_examples.additional_set_1`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.3_shot.all_examples.additional_set_2`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.3_shot.all_examples.witness_set`<br>`bigbench:simp_turing_concept.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>additional_set_1<br>additional_set_2<br>witness_set</details> | <details><summary>Description</summary><br>THE TASK MUST BE EVALUATED ONLY IN THE ZERO-SHOT SETTING Learning of algorithmic concepts in the universal language (Turing-complete) P3 under the perspective of machine teaching</details> | Generative, Scoring
[simple_arithmetic_json_subtasks](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/simple_arithmetic_json_subtasks/) | <details><summary>Tasks</summary><br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.0_shot.10_examples.one_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.0_shot.10_examples.three_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.0_shot.10_examples.two_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.0_shot.30_examples`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.0_shot.all_examples.one_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.0_shot.all_examples.three_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.0_shot.all_examples.two_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.1_shot.10_examples.one_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.1_shot.10_examples.three_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.1_shot.10_examples.two_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.1_shot.30_examples`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.1_shot.all_examples.one_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.1_shot.all_examples.three_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.1_shot.all_examples.two_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.2_shot.10_examples.one_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.2_shot.10_examples.three_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.2_shot.10_examples.two_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.2_shot.30_examples`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.2_shot.all_examples.one_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.2_shot.all_examples.three_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.2_shot.all_examples.two_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.3_shot.10_examples.one_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.3_shot.10_examples.three_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.3_shot.10_examples.two_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.3_shot.30_examples`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.3_shot.all_examples.one_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.3_shot.all_examples.three_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.3_shot.all_examples.two_digit`<br>`bigbench:simple_arithmetic_json_subtasks.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>one_digit<br>three_digit<br>two_digit</details> | <details><summary>Description</summary><br>A template task to be used as an example during task development (simple variable-difficulty arithmetic)</details> | Generative, Scoring
[strange_stories](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/strange_stories/) | <details><summary>Tasks</summary><br>`bigbench:strange_stories.mix.t5_default_vocab.0_shot.174_examples`<br>`bigbench:strange_stories.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:strange_stories.mix.t5_default_vocab.1_shot.174_examples`<br>`bigbench:strange_stories.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:strange_stories.mix.t5_default_vocab.2_shot.174_examples`<br>`bigbench:strange_stories.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:strange_stories.mix.t5_default_vocab.3_shot.174_examples`<br>`bigbench:strange_stories.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>boolean<br>multiple_choice</details> | <details><summary>Description</summary><br>A psychology test with naturalistic short stories that measures social intelligence Boolean options</details> | Multiple-Choice
[symbol_interpretation](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/symbol_interpretation/) | <details><summary>Tasks</summary><br>`bigbench:symbol_interpretation.mix.t5_default_vocab.0_shot.990_examples`<br>`bigbench:symbol_interpretation.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:symbol_interpretation.mix.t5_default_vocab.1_shot.990_examples`<br>`bigbench:symbol_interpretation.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:symbol_interpretation.mix.t5_default_vocab.2_shot.990_examples`<br>`bigbench:symbol_interpretation.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:symbol_interpretation.mix.t5_default_vocab.3_shot.990_examples`<br>`bigbench:symbol_interpretation.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>adversarial<br>emoji_agnostic<br>name_agnostic<br>plain<br>tricky</details> | <details><summary>Description</summary><br>Choose the sentence consistent with two given structures, where a structure is a sequence of six pieces represented by emojis</details> | Multiple-Choice
[tracking_shuffled_objects](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/tracking_shuffled_objects/) | <details><summary>Tasks</summary><br>`bigbench:tracking_shuffled_objects.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:tracking_shuffled_objects.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:tracking_shuffled_objects.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:tracking_shuffled_objects.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:tracking_shuffled_objects.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:tracking_shuffled_objects.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:tracking_shuffled_objects.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:tracking_shuffled_objects.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>five_objects<br>seven_objects<br>three_objects</details> | <details><summary>Description</summary><br>Determine the final positions of a set of objects given their initial positions and a description of a sequence of swaps</details> | Multiple-Choice
[undo_permutation](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/undo_permutation/) | <details><summary>Tasks</summary><br>`bigbench:undo_permutation.mix.t5_default_vocab.0_shot.300_examples`<br>`bigbench:undo_permutation.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:undo_permutation.mix.t5_default_vocab.1_shot.300_examples`<br>`bigbench:undo_permutation.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:undo_permutation.mix.t5_default_vocab.2_shot.300_examples`<br>`bigbench:undo_permutation.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:undo_permutation.mix.t5_default_vocab.3_shot.300_examples`<br>`bigbench:undo_permutation.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>reorder_byswap<br>reorder_chars<br>reorder_words</details> | <details><summary>Description</summary><br>Reorder scrambled words or sentences</details> | Multiple-Choice
[unit_conversion](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/unit_conversion/) | <details><summary>Tasks</summary><br>`bigbench:unit_conversion.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:unit_conversion.mix.t5_default_vocab.0_shot.128_examples.unit_addition_general`<br>`bigbench:unit_conversion.mix.t5_default_vocab.0_shot.128_examples.unit_addition_same`<br>`bigbench:unit_conversion.mix.t5_default_vocab.0_shot.128_examples.unit_conversion_with_relative_error`<br>`bigbench:unit_conversion.mix.t5_default_vocab.0_shot.all_examples.unit_addition_general`<br>`bigbench:unit_conversion.mix.t5_default_vocab.0_shot.all_examples.unit_addition_same`<br>`bigbench:unit_conversion.mix.t5_default_vocab.0_shot.all_examples.unit_conversion_with_relative_error`<br>`bigbench:unit_conversion.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:unit_conversion.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:unit_conversion.mix.t5_default_vocab.1_shot.128_examples.unit_addition_general`<br>`bigbench:unit_conversion.mix.t5_default_vocab.1_shot.128_examples.unit_addition_same`<br>`bigbench:unit_conversion.mix.t5_default_vocab.1_shot.128_examples.unit_conversion_with_relative_error`<br>`bigbench:unit_conversion.mix.t5_default_vocab.1_shot.all_examples.unit_addition_general`<br>`bigbench:unit_conversion.mix.t5_default_vocab.1_shot.all_examples.unit_addition_same`<br>`bigbench:unit_conversion.mix.t5_default_vocab.1_shot.all_examples.unit_conversion_with_relative_error`<br>`bigbench:unit_conversion.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:unit_conversion.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:unit_conversion.mix.t5_default_vocab.2_shot.128_examples.unit_addition_general`<br>`bigbench:unit_conversion.mix.t5_default_vocab.2_shot.128_examples.unit_addition_same`<br>`bigbench:unit_conversion.mix.t5_default_vocab.2_shot.128_examples.unit_conversion_with_relative_error`<br>`bigbench:unit_conversion.mix.t5_default_vocab.2_shot.all_examples.unit_addition_general`<br>`bigbench:unit_conversion.mix.t5_default_vocab.2_shot.all_examples.unit_addition_same`<br>`bigbench:unit_conversion.mix.t5_default_vocab.2_shot.all_examples.unit_conversion_with_relative_error`<br>`bigbench:unit_conversion.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:unit_conversion.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:unit_conversion.mix.t5_default_vocab.3_shot.128_examples.unit_addition_general`<br>`bigbench:unit_conversion.mix.t5_default_vocab.3_shot.128_examples.unit_addition_same`<br>`bigbench:unit_conversion.mix.t5_default_vocab.3_shot.128_examples.unit_conversion_with_relative_error`<br>`bigbench:unit_conversion.mix.t5_default_vocab.3_shot.all_examples.unit_addition_general`<br>`bigbench:unit_conversion.mix.t5_default_vocab.3_shot.all_examples.unit_addition_same`<br>`bigbench:unit_conversion.mix.t5_default_vocab.3_shot.all_examples.unit_conversion_with_relative_error`<br>`bigbench:unit_conversion.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>chained_conversion<br>different_systems<br>novel_systems<br>si_conversion<br>unit_addition_general<br>unit_addition_same<br>unit_conversion_with_relative_error<br>unit_identification</details> | <details><summary>Description</summary><br>Perform various tasks relating to units, including identification and conversion</details> | Generative, Multiple-Choice, Scoring
[unit_interpretation](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/unit_interpretation/) | <details><summary>Tasks</summary><br>`bigbench:unit_interpretation.mix.t5_default_vocab.0_shot.100_examples`<br>`bigbench:unit_interpretation.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:unit_interpretation.mix.t5_default_vocab.1_shot.100_examples`<br>`bigbench:unit_interpretation.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:unit_interpretation.mix.t5_default_vocab.2_shot.100_examples`<br>`bigbench:unit_interpretation.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:unit_interpretation.mix.t5_default_vocab.3_shot.100_examples`<br>`bigbench:unit_interpretation.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>lv0<br>lv1<br>lv2<br>lv3</details> | <details><summary>Description</summary><br>Solve arithmetic problems involving quantities with units</details> | Multiple-Choice
[unnatural_in_context_learning](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/unnatural_in_context_learning/) | <details><summary>Tasks</summary><br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.1024_examples`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.128_examples.dates_unnatural_content_and_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.128_examples.dates_unnatural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.128_examples.dates_unnatural_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.128_examples.dates`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.128_examples.identity`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.128_examples.reverse_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.128_examples.reverse_to_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.128_examples.unnatural_addition_2_digit`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.all_examples.dates_unnatural_content_and_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.all_examples.dates_unnatural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.all_examples.dates_unnatural_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.all_examples.dates`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.all_examples.identity`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.all_examples.reverse_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.all_examples.reverse_to_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.all_examples.unnatural_addition_2_digit`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.0_shot.all_examples`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.1024_examples`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.128_examples.dates_unnatural_content_and_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.128_examples.dates_unnatural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.128_examples.dates_unnatural_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.128_examples.dates`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.128_examples.identity`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.128_examples.reverse_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.128_examples.reverse_to_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.128_examples.unnatural_addition_2_digit`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.all_examples.dates_unnatural_content_and_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.all_examples.dates_unnatural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.all_examples.dates_unnatural_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.all_examples.dates`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.all_examples.identity`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.all_examples.reverse_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.all_examples.reverse_to_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.all_examples.unnatural_addition_2_digit`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.1_shot.all_examples`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.1024_examples`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.128_examples.dates_unnatural_content_and_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.128_examples.dates_unnatural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.128_examples.dates_unnatural_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.128_examples.dates`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.128_examples.identity`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.128_examples.reverse_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.128_examples.reverse_to_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.128_examples.unnatural_addition_2_digit`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.all_examples.dates_unnatural_content_and_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.all_examples.dates_unnatural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.all_examples.dates_unnatural_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.all_examples.dates`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.all_examples.identity`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.all_examples.reverse_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.all_examples.reverse_to_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.all_examples.unnatural_addition_2_digit`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.2_shot.all_examples`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.1024_examples`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.128_examples.dates_unnatural_content_and_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.128_examples.dates_unnatural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.128_examples.dates_unnatural_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.128_examples.dates`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.128_examples.identity`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.128_examples.reverse_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.128_examples.reverse_to_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.128_examples.unnatural_addition_2_digit`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.all_examples.dates_unnatural_content_and_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.all_examples.dates_unnatural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.all_examples.dates_unnatural_form`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.all_examples.dates`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.all_examples.identity`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.all_examples.reverse_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.all_examples.reverse_to_natural_content`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.all_examples.unnatural_addition_2_digit`<br>`bigbench:unnatural_in_context_learning.mix.t5_default_vocab.3_shot.all_examples`</details>  | <details><summary>Subtasks</summary><br>dates<br>dates_unnatural_content<br>dates_unnatural_content_and_form<br>dates_unnatural_form<br>identity<br>reverse_natural_content<br>reverse_to_natural_content<br>unnatural_addition_2_digit</details> | <details><summary>Description</summary><br>Perform in-context learning using context sequences which have been modified so as to likely be out of the training distribution, and require the model to attend to unnatural in-context patterns</details> | Generative, Scoring