KNLOV5     €   KNLOSB1        ÷¤      w      Æû‡]ù!.ÎHdàcð;ã¥—I±Bí<Ã¿ê‡yþÏZŒ£@'Eÿ¥ÌÕl2Úþ÷Ý	jÊa·|3)y:òl\üDêÌûsá
”¸A%÷Uãª_ÃTÅdüB¥·                                                                                                                                KSEG  \_      +[
–kÀ;H<C
Ÿ„vvsù½ÈEíêŸñÙ˜¤ebytesY¡Temperature in eval: freeze it. Compare prompts at the same temperature and top-p you will ship. A creative temperature that looks better on a judge and worse on exact JSON is not an improvement.

For deterministic gates (schema, tool names), use temperature 0. For pass@k research, report the sampling settings.

Seeds help, but they do not make a hosted API fully deterministic. Repeat the run if the gate is close.bidxGsha256-1269f143ca81e88d69a3cda8c503118d2cad496399b04bb33cd1ba3ef9e4d77adkindechunkdmeta¦edocIdktemperaturegheadingsTemperature in evalmlegacyBlockIdinamespacehsamplinglsourceObjectxGsha256-5b618f5ebf98d8e999d1ce8ef22f7682ae636c6c0d22333b6cb3fb2c18a0516adspan¢cend¡estart ¤ebytesYLLM as judge: a second model grades the first against a rubric. It is cheap and noisy. Calibrate it against a human sample before you trust a 2-point swing.

Judges favor verbose, confident answers and models from the same family. Blind the judge to which system produced the answer. Shuffle order in pairwise tests.

A judge without a written rubric is a vibe. Write the rubric first.bidxGsha256-199e68f52fe6267863033be1fbf410457f595de26301c16945cd39ea9c3cc5bcdkindfsourcedmeta£edocIdlllm-as-judgemlegacyBlockIdinamespacefjudges¤ebytesXf¢lsourceDigestxGsha256-a5e09cc426ac316af928f54ff195a8ece19c340acaf669b37514b725e6427ffemsourceVersionbidxGsha256-26a0a1979142153bdebf975077f1e8085bfb9478cba217d053d3520a1e3fa1d5dkindhmetadatadmeta¢lsourceDigestxGsha256-a5e09cc426ac316af928f54ff195a8ece19c340acaf669b37514b725e6427ffemsourceVersion¤ebytesY}prompt regression: treat the prompt like code. A change that drops golden-set scores, raises refusals on allowed tasks, or breaks JSON schema fails CI.

Keep the previous prompt in git. Diff scores by slice (language, tool-use, safety) not one average that hides a collapse.

Anecdotes from two teammates are not a regression test. They are a reason to add cases to the golden set.bidxGsha256-295970d2bd99406939af85ee9c101f006c4d5b5a4c97d2002db88911c6282b6edkindechunkdmeta¦edocIdqprompt-regressiongheadingqprompt regressionmlegacyBlockIdinamespacejregressionlsourceObjectxGsha256-b94140f7f2ea439846339d2e97dd13fdd0183520dbe647bd201abcffc8bf6ac9dspan¢cend}estart ¤ebytesYuJudge pitfalls: position bias (first answer wins), length bias, self-preference, and rubric leakage into the generator prompt.

Mitigations: swap order, cap length in the rubric, use a different model family as judge, and keep the generator from seeing the judge's scoring notes.

If humans disagree with the judge on more than a thin slice, stop using the judge as a gate.bidxGsha256-44c81fe219407e78439cadc5452a71277de39c16cda04133678fb511b96831dddkindechunkdmeta¦edocIdnjudge-pitfallsgheadingnJudge pitfallsmlegacyBlockIdinamespacefjudgeslsourceObjectxGsha256-fd28669e76998f27f651dcd2f1fa212be13f03ba25216a2014df4560d4299199dspan¢cenduestart ¤ebytesY¤pass@k: the chance that at least one of k samples passes a check. It is the right metric for "can the model produce a correct program", not for a single-shot customer reply.

Do not report pass@1 from a cherry-picked temperature. State k, temperature, and the checker.

For a production chat that sends one answer, pass@1 (or a graded score at k=1) is what the user sees. pass@8 is research unless you actually sample 8.bidxGsha256-52845ec0378480186cb7c39a6827fe61219d806a45422b2af7f16c3de0d11488dkindfsourcedmeta£edocIdipass-at-kmlegacyBlockIdinamespacegmetrics¤ebytesY¡Temperature in eval: freeze it. Compare prompts at the same temperature and top-p you will ship. A creative temperature that looks better on a judge and worse on exact JSON is not an improvement.

For deterministic gates (schema, tool names), use temperature 0. For pass@k research, report the sampling settings.

Seeds help, but they do not make a hosted API fully deterministic. Repeat the run if the gate is close.bidxGsha256-5b618f5ebf98d8e999d1ce8ef22f7682ae636c6c0d22333b6cb3fb2c18a0516adkindfsourcedmeta£edocIdktemperaturemlegacyBlockIdinamespacehsampling¤ebytesYÉ{"version":1,"nodes":[{"id":"n_13151c55","label":"few-shot examples that are near-copies of test items"},{"id":"n_166838cd","label":"a correct final sentence after a wrong tool call"},{"id":"n_175be3ae","label":"a model that refuses invoicing questions"},{"id":"n_1f3a7fc9","label":"a prompt tweak that improves coding and starts answering the disallowed slice"},{"id":"n_23711fd1","label":"a regression"},{"id":"n_27cb3b23","label":"close"},{"id":"n_2ef9aea6","label":"a bad golden source unless you freeze it"},{"id":"n_317f06af","label":"safety eval"},{"id":"n_3d3fd328","label":"a failed release"},{"id":"n_3de052b8","label":"normal hard checks for structure a rubric for the rest"},{"id":"n_41b03ce3","label":"not an independent rater"},{"id":"n_43a00f8a","label":"a creative temperature that looks better on a judge and worse on exact json"},{"id":"n_4aa7772c","label":"not safer for an invoicing product"},{"id":"n_52556354","label":"a pack like this one"},{"id":"n_528513ce","label":"the right kind of evidence stable ids headings and text"},{"id":"n_5678cfaa","label":"not a regression test"},{"id":"n_607b4f11","label":"leakage"},{"id":"n_675b3b67","label":"a fluent wrong answer"},{"id":"n_702f831a","label":"a hybrid"},{"id":"n_7692e651","label":"not an improvement"},{"id":"n_8551cb14","label":"do not use a web search during grading unless the product"},{"id":"n_8a05d516","label":"still a fail if the product billed the tool"},{"id":"n_8c053a1b","label":"repeat the run if the gate"},{"id":"n_8fa2ecac","label":"a chat transcript"},{"id":"n_90d8e7c1","label":"worse than a refusal"},{"id":"n_aabc8a34","label":"anecdotes from two teammates"},{"id":"n_b461eaa1","label":"disagreement"},{"id":"n_b5e750a8","label":"a vibe"},{"id":"n_b7e9a6c4","label":"allowed to search"},{"id":"n_bbdf8d76","label":"over-calling"},{"id":"n_c8916050","label":"a judge without a written rubric"},{"id":"n_d33b0a07","label":"cheapest to fix in the rubric not in the model"},{"id":"n_e46023c5","label":"a single annotator who also wrote the prompt"},{"id":"n_f572c6df","label":"not a substitute for product policy"}],"edges":[{"id":"e_033a524b","from":"n_e46023c5","p":"is","to":"n_41b03ce3","evidence":[9]},{"id":"e_066345c0","from":"n_bbdf8d76","p":"is","to":"n_23711fd1","evidence":[10]},{"id":"e_1f818c38","from":"n_175be3ae","p":"is","to":"n_4aa7772c","evidence":[13]},{"id":"e_2cdd0771","from":"n_b461eaa1","p":"is","to":"n_d33b0a07","evidence":[9]},{"id":"e_438278e4","from":"n_1f3a7fc9","p":"is","to":"n_3d3fd328","evidence":[13]},{"id":"e_7517654b","from":"n_702f831a","p":"is","to":"n_3de052b8","evidence":[1]},{"id":"e_753e1312","from":"n_52556354","p":"is","to":"n_528513ce","evidence":[11]},{"id":"e_8b871e9d","from":"n_43a00f8a","p":"is","to":"n_7692e651","evidence":[8]},{"id":"e_9d6a3b89","from":"n_13151c55","p":"is","to":"n_607b4f11","evidence":[7]},{"id":"e_a7d9c233","from":"n_aabc8a34","p":"is","to":"n_5678cfaa","evidence":[6]},{"id":"e_ab2fc002","from":"n_8c053a1b","p":"is","to":"n_27cb3b23","evidence":[8]},{"id":"e_acea65cc","from":"n_317f06af","p":"is","to":"n_f572c6df","evidence":[13]},{"id":"e_b246a8ab","from":"n_8fa2ecac","p":"is","to":"n_2ef9aea6","evidence":[11]},{"id":"e_bce68900","from":"n_8551cb14","p":"is","to":"n_b7e9a6c4","evidence":[5]},{"id":"e_ca5af217","from":"n_675b3b67","p":"is","to":"n_90d8e7c1","evidence":[5]},{"id":"e_d6f35c9a","from":"n_c8916050","p":"is","to":"n_b5e750a8","evidence":[2]},{"id":"e_e4b607bb","from":"n_166838cd","p":"is","to":"n_8a05d516","evidence":[10]}],"index":{"labelToId":{"few-shot examples that are near-copies of test items":"n_13151c55","a correct final sentence after a wrong tool call":"n_166838cd","a model that refuses invoicing questions":"n_175be3ae","a prompt tweak that improves coding and starts answering the disallowed slice":"n_1f3a7fc9","a regression":"n_23711fd1","close":"n_27cb3b23","a bad golden source unless you freeze it":"n_2ef9aea6","safety eval":"n_317f06af","a failed release":"n_3d3fd328","normal hard checks for structure a rubric for the rest":"n_3de052b8","not an independent rater":"n_41b03ce3","a creative temperature that looks better on a judge and worse on exact json":"n_43a00f8a","not safer for an invoicing product":"n_4aa7772c","a pack like this one":"n_52556354","the right kind of evidence stable ids headings and text":"n_528513ce","not a regression test":"n_5678cfaa","leakage":"n_607b4f11","a fluent wrong answer":"n_675b3b67","a hybrid":"n_702f831a","not an improvement":"n_7692e651","do not use a web search during grading unless the product":"n_8551cb14","still a fail if the product billed the tool":"n_8a05d516","repeat the run if the gate":"n_8c053a1b","a chat transcript":"n_8fa2ecac","worse than a refusal":"n_90d8e7c1","anecdotes from two teammates":"n_aabc8a34","disagreement":"n_b461eaa1","a vibe":"n_b5e750a8","allowed to search":"n_b7e9a6c4","over-calling":"n_bbdf8d76","a judge without a written rubric":"n_c8916050","cheapest to fix in the rubric not in the model":"n_d33b0a07","a single annotator who also wrote the prompt":"n_e46023c5","not a substitute for product policy":"n_f572c6df"},"out":{"n_e46023c5":["e_033a524b"],"n_bbdf8d76":["e_066345c0"],"n_175be3ae":["e_1f818c38"],"n_b461eaa1":["e_2cdd0771"],"n_1f3a7fc9":["e_438278e4"],"n_702f831a":["e_7517654b"],"n_52556354":["e_753e1312"],"n_43a00f8a":["e_8b871e9d"],"n_13151c55":["e_9d6a3b89"],"n_aabc8a34":["e_a7d9c233"],"n_8c053a1b":["e_ab2fc002"],"n_317f06af":["e_acea65cc"],"n_8fa2ecac":["e_b246a8ab"],"n_8551cb14":["e_bce68900"],"n_675b3b67":["e_ca5af217"],"n_c8916050":["e_d6f35c9a"],"n_166838cd":["e_e4b607bb"]},"in":{"n_41b03ce3":["e_033a524b"],"n_23711fd1":["e_066345c0"],"n_4aa7772c":["e_1f818c38"],"n_d33b0a07":["e_2cdd0771"],"n_3d3fd328":["e_438278e4"],"n_3de052b8":["e_7517654b"],"n_528513ce":["e_753e1312"],"n_7692e651":["e_8b871e9d"],"n_607b4f11":["e_9d6a3b89"],"n_5678cfaa":["e_a7d9c233"],"n_27cb3b23":["e_ab2fc002"],"n_f572c6df":["e_acea65cc"],"n_2ef9aea6":["e_b246a8ab"],"n_b7e9a6c4":["e_bce68900"],"n_90d8e7c1":["e_ca5af217"],"n_b5e750a8":["e_d6f35c9a"],"n_8a05d516":["e_e4b607bb"]}}}bidxGsha256-60828128a93bb2881659d6ac67d1a24c27189252d8580c3b019597281b8c1d59dkindfclaimsdmeta¢hencodinggjson-v4gversion¤ebytesY•train test leakage: if the eval items were in the fine-tune data, the prompt, the few-shot block, or a public benchmark the model memorized, the score is contaminated.

Build a private holdout. Rotate a slice of production tickets that the prompt author has not seen. Mark public benchmarks as "reported, not gated".

Few-shot examples that are near-copies of test items are leakage. Check n-gram overlap.bidxGsha256-6674db10ebd937c366de80e51079b07c752a7eedb58a28f997bfc44060376da8dkindfsourcedmeta£edocIdrtrain-test-leakagemlegacyBlockIdinamespacegleakage¤ebytesYlExact match vs semantic: use exact match or regex for IDs, JSON keys, tool names, and "must include this statute". Use a graded rubric or a judge for free prose.

Exact match on a whole paragraph punishes harmless rephrasing and rewards lucky wording. Do not use it as the only score for chat.

A hybrid is normal: hard checks for structure, a rubric for the rest.bidxGsha256-6851502b2d1f262e32cf009ceb24d5bb9d42231531f185bf0fc8c7e1bd5e5080dkindfsourcedmeta£edocIdqexact-vs-semanticmlegacyBlockIdinamespacegmetrics¤ebytesYXTool-use eval: score the tool name, arguments, and whether the tool should have been called at all. A correct final sentence after a wrong tool call is still a fail if the product billed the tool.

Golden items should include "must not call tools" cases. Over-calling is a regression.

Stub tools in CI. Do not hit live APIs from the eval loop.bidxGsha256-68b4c62932cb920ef5928dc7ba7abbe90eb1ca0c50e9ba90f166becc4492fe4ddkindfsourcedmeta£edocIdmtool-use-evalmlegacyBlockId
inamespacegmetrics¤ebytesYXTool-use eval: score the tool name, arguments, and whether the tool should have been called at all. A correct final sentence after a wrong tool call is still a fail if the product billed the tool.

Golden items should include "must not call tools" cases. Over-calling is a regression.

Stub tools in CI. Do not hit live APIs from the eval loop.bidxGsha256-68bbf7f3d5738b01e2d6620b1e944287987fd9cefcb5b79c1fbffb1785a09cbedkindechunkdmeta¦edocIdmtool-use-evalgheadingmTool-use evalmlegacyBlockId
inamespacegmetricslsourceObjectxGsha256-68b4c62932cb920ef5928dc7ba7abbe90eb1ca0c50e9ba90f166becc4492fe4ddspan¢cendXestart ¤ebytesYlExact match vs semantic: use exact match or regex for IDs, JSON keys, tool names, and "must include this statute". Use a graded rubric or a judge for free prose.

Exact match on a whole paragraph punishes harmless rephrasing and rewards lucky wording. Do not use it as the only score for chat.

A hybrid is normal: hard checks for structure, a rubric for the rest.bidxGsha256-7579fd41df4c7a3e793bc04e281a0f2f1fbb31798f3676336bb2c0b8b0fbf369dkindechunkdmeta¦edocIdqexact-vs-semanticgheadingwExact match vs semanticmlegacyBlockIdinamespacegmetricslsourceObjectxGsha256-6851502b2d1f262e32cf009ceb24d5bb9d42231531f185bf0fc8c7e1bd5e5080dspan¢cendlestart ¤ebytesYÍHallucination checks: require claims to be supported by retrieved evidence. Score "unsupported sentence" separately from "wrong fact". A fluent wrong answer is worse than a refusal.

For retrieval systems, the eval input includes the corpus slice the system should have seen. If you only score the final sentence, you miss citation errors.

Do not use a web search during grading unless the product is allowed to search. You would be scoring a different system.bidxGsha256-8dd696b33bd97df0e4bedf1ab201b5f8e3f2e2043dc8ec4b98206ac2311bfe02dkindfsourcedmeta£edocIdshallucination-checkmlegacyBlockIdinamespacegmetrics¤ebytesY•train test leakage: if the eval items were in the fine-tune data, the prompt, the few-shot block, or a public benchmark the model memorized, the score is contaminated.

Build a private holdout. Rotate a slice of production tickets that the prompt author has not seen. Mark public benchmarks as "reported, not gated".

Few-shot examples that are near-copies of test items are leakage. Check n-gram overlap.bidxGsha256-93e5a6204d496d066bbf8d7598341d82bc406ab9bed28360eb7ec82fa87f676edkindechunkdmeta¦edocIdrtrain-test-leakagegheadingrtrain test leakagemlegacyBlockIdinamespacegleakagelsourceObjectxGsha256-6674db10ebd937c366de80e51079b07c752a7eedb58a28f997bfc44060376da8dspan¢cend•estart ¤ebytesY¤pass@k: the chance that at least one of k samples passes a check. It is the right metric for "can the model produce a correct program", not for a single-shot customer reply.

Do not report pass@1 from a cherry-picked temperature. State k, temperature, and the checker.

For a production chat that sends one answer, pass@1 (or a graded score at k=1) is what the user sees. pass@8 is research unless you actually sample 8.bidxGsha256-9685aa28c4ddb7f261b49c614a03f70e0509b4069dab1ebda3e6f820d1c2d889dkindechunkdmeta¦edocIdipass-at-kgheadingfpass@kmlegacyBlockIdinamespacegmetricslsourceObjectxGsha256-52845ec0378480186cb7c39a6827fe61219d806a45422b2af7f16c3de0d11488dspan¢cend¤estart ¤ebytesY^{"version":1,"agents":[{"id":"prompt-eval-coach","version":1,"name":"Prompt Eval Coach","description":"Answers prompt evaluation questions from this pack. Never calls a model or a shell.","systemPrompt":["Answer only from retrieved evidence.","Name the metric and whether a human still has to review.","Do not invent benchmark numbers. If the pack does not cover it, say so.","Do not execute a shell, model API, or any tool."],"retrievalDefaults":{"namespace":["goldens","judges","metrics","leakage","regression","sampling"],"topK":5},"toolPolicy":{"mode":"deny","tools":["shell","exec","model","http"]}}]}bidxGsha256-a204257b87ce4df422fc19fefbc7a2433767b5f8aca8cabaeddecd3dfb632b1adkindfagentsdmeta¢hencodinggjson-v4gversion¤ebytesYŽCitation grounding: if the product shows sources, every cited id must exist in the retrieved set, and the sentence must be supported by that source.

Score missing citations, extra citations, and wrong citations separately. Users click the wrong one.

A pack like this one is the right kind of evidence: stable ids, headings, and text. A chat transcript is a bad golden source unless you freeze it.bidxGsha256-af8cf48f258264ed133d4facec028e4316f2b1a5f6d67cdcd6a6faa7e551a9d1dkindfsourcedmeta£edocIdrcitation-groundingmlegacyBlockIdinamespacegmetrics¤ebytesY¶golden set: a frozen list of inputs with expected properties â€” exact strings, required citations, forbidden claims, or a human grade. You run the system against it on every prompt or model change.

Start small and real: 50â€“200 tasks from production-like traffic, not puzzles from a blog. Cover the failures you actually see.

If you edit the golden set in the same PR as the prompt, you are not measuring the prompt. Split the change.bidxGsha256-b3227935b5e3b03b12407fa3d0223b0a6d6e1839c6882b05b9c6eb07e34882cfdkindechunkdmeta¦edocIdjgolden-setgheadingjgolden setmlegacyBlockId inamespaceggoldenslsourceObjectxGsha256-bb2b61bc7b0ee5132ed1fe147d1d48f8f9289ab1cdb40e13b82a09b59cda3ab2dspan¢cend²estart ¤ebytesYÍHallucination checks: require claims to be supported by retrieved evidence. Score "unsupported sentence" separately from "wrong fact". A fluent wrong answer is worse than a refusal.

For retrieval systems, the eval input includes the corpus slice the system should have seen. If you only score the final sentence, you miss citation errors.

Do not use a web search during grading unless the product is allowed to search. You would be scoring a different system.bidxGsha256-b636499925ad5d1f56f96bab8cfe2f97edc6f48ef063ee26fa53b44a70c4efe7dkindechunkdmeta¦edocIdshallucination-checkgheadingtHallucination checksmlegacyBlockIdinamespacegmetricslsourceObjectxGsha256-8dd696b33bd97df0e4bedf1ab201b5f8e3f2e2043dc8ec4b98206ac2311bfe02dspan¢cendÍestart ¤ebytesY}prompt regression: treat the prompt like code. A change that drops golden-set scores, raises refusals on allowed tasks, or breaks JSON schema fails CI.

Keep the previous prompt in git. Diff scores by slice (language, tool-use, safety) not one average that hides a collapse.

Anecdotes from two teammates are not a regression test. They are a reason to add cases to the golden set.bidxGsha256-b94140f7f2ea439846339d2e97dd13fdd0183520dbe647bd201abcffc8bf6ac9dkindfsourcedmeta£edocIdqprompt-regressionmlegacyBlockIdinamespacejregression¤ebytesY¶golden set: a frozen list of inputs with expected properties â€” exact strings, required citations, forbidden claims, or a human grade. You run the system against it on every prompt or model change.

Start small and real: 50â€“200 tasks from production-like traffic, not puzzles from a blog. Cover the failures you actually see.

If you edit the golden set in the same PR as the prompt, you are not measuring the prompt. Split the change.bidxGsha256-bb2b61bc7b0ee5132ed1fe147d1d48f8f9289ab1cdb40e13b82a09b59cda3ab2dkindfsourcedmeta£edocIdjgolden-setmlegacyBlockId inamespaceggoldens¤ebytesYŽCitation grounding: if the product shows sources, every cited id must exist in the retrieved set, and the sentence must be supported by that source.

Score missing citations, extra citations, and wrong citations separately. Users click the wrong one.

A pack like this one is the right kind of evidence: stable ids, headings, and text. A chat transcript is a bad golden source unless you freeze it.bidxGsha256-bd59e9ebb72d5be196557cc0801621c19650f35d88918d8ea80e105b6ca1c4b2dkindechunkdmeta¦edocIdrcitation-groundinggheadingrCitation groundingmlegacyBlockIdinamespacegmetricslsourceObjectxGsha256-af8cf48f258264ed133d4facec028e4316f2b1a5f6d67cdcd6a6faa7e551a9d1dspan¢cendŽestart ¤ebytesY†Safety slice: keep a frozen set of disallowed and allowed tasks. A prompt tweak that improves coding and starts answering the disallowed slice is a failed release.

Measure over-refusal too. A model that refuses invoicing questions is not "safer" for an invoicing product.

Safety eval is not a substitute for product policy. It only checks that the prompt still obeys the policy you wrote.bidxGsha256-d499a8116f750bf7376e2a9478dc1796b8ace5ac28ccf7c0b33b1cf1993e23e3dkindechunkdmeta¦edocIdlsafety-slicegheadinglSafety slicemlegacyBlockIdinamespacejregressionlsourceObjectxGsha256-f2e928e124d7ded8825a963728e80fffe3a9c733fb43bcc861098bea268f8719dspan¢cend†estart ¤ebytesY^Inter-rater agreement: two humans grade a sample of the golden set. If they disagree, the rubric is unclear and a judge cannot save you.

Measure agreement (simple percent or Cohen's kappa) before you automate. Disagreement is cheapest to fix in the rubric, not in the model.

A single annotator who also wrote the prompt is not an independent rater.bidxGsha256-d4c6f5fffcd9231ce88d5bb4089e52ca46af47f180146af58c6a0b5ee33018f9dkindfsourcedmeta£edocIdkinter-ratermlegacyBlockId	inamespaceggoldens¤ebytesYLLM as judge: a second model grades the first against a rubric. It is cheap and noisy. Calibrate it against a human sample before you trust a 2-point swing.

Judges favor verbose, confident answers and models from the same family. Blind the judge to which system produced the answer. Shuffle order in pairwise tests.

A judge without a written rubric is a vibe. Write the rubric first.bidxGsha256-d987f42c7c59dbb3e558d5d848fa31c4f64066b36941bd4aec2b6ff6cfc8f53ddkindechunkdmeta¦edocIdlllm-as-judgegheadinglLLM as judgemlegacyBlockIdinamespacefjudgeslsourceObjectxGsha256-199e68f52fe6267863033be1fbf410457f595de26301c16945cd39ea9c3cc5bcdspan¢cendestart ¤ebytesYHCost of eval: a gate that costs more than the product run will be skipped. Keep the CI set small and fast. Park the large set on nightly.

Cache model outputs keyed by prompt hash plus input id so a rubric-only change does not re-generate.

If you cannot afford to run eval, you cannot afford to change the prompt in production.bidxGsha256-e9ac6c2f737b8b2edc9a9a87a020f0c9a5a94339bff2dcb7e995d8225b6fa10fdkindfsourcedmeta£edocIdieval-costmlegacyBlockIdinamespacejregression¤ebytesYHCost of eval: a gate that costs more than the product run will be skipped. Keep the CI set small and fast. Park the large set on nightly.

Cache model outputs keyed by prompt hash plus input id so a rubric-only change does not re-generate.

If you cannot afford to run eval, you cannot afford to change the prompt in production.bidxGsha256-ea915db8b2929ba53175b6ef6fd9152fe6074f37b37c9b6943ecd8455d1983d2dkindechunkdmeta¦edocIdieval-costgheadinglCost of evalmlegacyBlockIdinamespacejregressionlsourceObjectxGsha256-e9ac6c2f737b8b2edc9a9a87a020f0c9a5a94339bff2dcb7e995d8225b6fa10fdspan¢cendHestart ¤ebytesY†Safety slice: keep a frozen set of disallowed and allowed tasks. A prompt tweak that improves coding and starts answering the disallowed slice is a failed release.

Measure over-refusal too. A model that refuses invoicing questions is not "safer" for an invoicing product.

Safety eval is not a substitute for product policy. It only checks that the prompt still obeys the policy you wrote.bidxGsha256-f2e928e124d7ded8825a963728e80fffe3a9c733fb43bcc861098bea268f8719dkindfsourcedmeta£edocIdlsafety-slicemlegacyBlockIdinamespacejregression¤ebytesY^Inter-rater agreement: two humans grade a sample of the golden set. If they disagree, the rubric is unclear and a judge cannot save you.

Measure agreement (simple percent or Cohen's kappa) before you automate. Disagreement is cheapest to fix in the rubric, not in the model.

A single annotator who also wrote the prompt is not an independent rater.bidxGsha256-f4095d3c583e7461badeed5bc8658f24e8c5f8d3ff645b70be11028a5fdb54ecdkindechunkdmeta¦edocIdkinter-ratergheadinguInter-rater agreementmlegacyBlockId	inamespaceggoldenslsourceObjectxGsha256-d4c6f5fffcd9231ce88d5bb4089e52ca46af47f180146af58c6a0b5ee33018f9dspan¢cend^estart ¤ebytesYuJudge pitfalls: position bias (first answer wins), length bias, self-preference, and rubric leakage into the generator prompt.

Mitigations: swap order, cap length in the rubric, use a different model family as judge, and keep the generator from seeing the judge's scoring notes.

If humans disagree with the judge on more than a thin slice, stop using the judge as a gate.bidxGsha256-fd28669e76998f27f651dcd2f1fa212be13f03ba25216a2014df4560d4299199dkindfsourcedmeta£edocIdnjudge-pitfallsmlegacyBlockIdinamespacefjudgesKSEG  +D      î½[Üf(©ÍžÌíÙ€&4³ƒ„˜[%š®Àxš˜ªeactorqknolo-v4-migratorlactorCounterbidxGsha256-0339fc0c72e10a76fc1dc1909888f0a89494995bea19003d14a5a126fbd59136dkindjsource.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-52845ec0378480186cb7c39a6827fe61219d806a45422b2af7f16c3de0d11488jprovenance¡hobjectIdxGsha256-52845ec0378480186cb7c39a6827fe61219d806a45422b2af7f16c3de0d11488ftargetxGsha256-52845ec0378480186cb7c39a6827fe61219d806a45422b2af7f16c3de0d11488mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-041214ff7c241dae769b97c0f96f4fc50d352927f58a3e5bca2a6e2a61c9b3d2dkindjsource.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-b94140f7f2ea439846339d2e97dd13fdd0183520dbe647bd201abcffc8bf6ac9jprovenance¡hobjectIdxGsha256-b94140f7f2ea439846339d2e97dd13fdd0183520dbe647bd201abcffc8bf6ac9ftargetxGsha256-b94140f7f2ea439846339d2e97dd13fdd0183520dbe647bd201abcffc8bf6ac9mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-0e479d6ce792b9da96385a518fde9f8669a5c42fbb18cd7c874494814d90026fdkindldocument.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-93e5a6204d496d066bbf8d7598341d82bc406ab9bed28360eb7ec82fa87f676ejprovenance¡hobjectIdxGsha256-93e5a6204d496d066bbf8d7598341d82bc406ab9bed28360eb7ec82fa87f676eftargetxGsha256-93e5a6204d496d066bbf8d7598341d82bc406ab9bed28360eb7ec82fa87f676emtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounter	bidxGsha256-123e67e46c9674d1c0580d62091e874ea2da62be595a4175ab4c9885c296d5aadkindjsource.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-6674db10ebd937c366de80e51079b07c752a7eedb58a28f997bfc44060376da8jprovenance¡hobjectIdxGsha256-6674db10ebd937c366de80e51079b07c752a7eedb58a28f997bfc44060376da8ftargetxGsha256-6674db10ebd937c366de80e51079b07c752a7eedb58a28f997bfc44060376da8mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-13f66847b8554cd7e6cfd9b10b5a5dd8c5530e9efaa0f323db4ccf237f3ccfc3dkindldocument.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-295970d2bd99406939af85ee9c101f006c4d5b5a4c97d2002db88911c6282b6ejprovenance¡hobjectIdxGsha256-295970d2bd99406939af85ee9c101f006c4d5b5a4c97d2002db88911c6282b6eftargetxGsha256-295970d2bd99406939af85ee9c101f006c4d5b5a4c97d2002db88911c6282b6emtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-18a48e2620e966d1dff68ed376833ef77be3b8770ccae912a58a0e6add0e16e8dkindjagents.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-a204257b87ce4df422fc19fefbc7a2433767b5f8aca8cabaeddecd3dfb632b1ajprovenance¡hobjectIdxGsha256-a204257b87ce4df422fc19fefbc7a2433767b5f8aca8cabaeddecd3dfb632b1aftargetxGsha256-a204257b87ce4df422fc19fefbc7a2433767b5f8aca8cabaeddecd3dfb632b1amtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-2241c8567d67eda91f36db1935c6c631dd93ae6cecd0975630f2c3b8dc36e6b1dkindlmetadata.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-26a0a1979142153bdebf975077f1e8085bfb9478cba217d053d3520a1e3fa1d5jprovenance¡hobjectIdxGsha256-26a0a1979142153bdebf975077f1e8085bfb9478cba217d053d3520a1e3fa1d5ftargetxGsha256-26a0a1979142153bdebf975077f1e8085bfb9478cba217d053d3520a1e3fa1d5mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-3bc799a22a2f48a3ae1583870b54afe5256bdfbe7e96e50a4a9da3fbf7c5cc3adkindjsource.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-5b618f5ebf98d8e999d1ce8ef22f7682ae636c6c0d22333b6cb3fb2c18a0516ajprovenance¡hobjectIdxGsha256-5b618f5ebf98d8e999d1ce8ef22f7682ae636c6c0d22333b6cb3fb2c18a0516aftargetxGsha256-5b618f5ebf98d8e999d1ce8ef22f7682ae636c6c0d22333b6cb3fb2c18a0516amtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-4e177d834d4ad68028e7716e2775b0a67d6559368b6219a62d19328b2bdb16abdkindjclaims.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-60828128a93bb2881659d6ac67d1a24c27189252d8580c3b019597281b8c1d59jprovenance¡hobjectIdxGsha256-60828128a93bb2881659d6ac67d1a24c27189252d8580c3b019597281b8c1d59ftargetxGsha256-60828128a93bb2881659d6ac67d1a24c27189252d8580c3b019597281b8c1d59mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-4f3483ce528d425fa3326025a3e87c707b4d1749cb17b0d43696013576ad8159dkindldocument.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-ea915db8b2929ba53175b6ef6fd9152fe6074f37b37c9b6943ecd8455d1983d2jprovenance¡hobjectIdxGsha256-ea915db8b2929ba53175b6ef6fd9152fe6074f37b37c9b6943ecd8455d1983d2ftargetxGsha256-ea915db8b2929ba53175b6ef6fd9152fe6074f37b37c9b6943ecd8455d1983d2mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-5987dcb167e940520a2ee986738f1dcde0d85de3d8f6d41475446f9ca9885b55dkindjsource.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-e9ac6c2f737b8b2edc9a9a87a020f0c9a5a94339bff2dcb7e995d8225b6fa10fjprovenance¡hobjectIdxGsha256-e9ac6c2f737b8b2edc9a9a87a020f0c9a5a94339bff2dcb7e995d8225b6fa10fftargetxGsha256-e9ac6c2f737b8b2edc9a9a87a020f0c9a5a94339bff2dcb7e995d8225b6fa10fmtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-5d0e343f37c4280cebfeef198274893941914932a67bcb92ce3406924e80a665dkindjsource.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-d4c6f5fffcd9231ce88d5bb4089e52ca46af47f180146af58c6a0b5ee33018f9jprovenance¡hobjectIdxGsha256-d4c6f5fffcd9231ce88d5bb4089e52ca46af47f180146af58c6a0b5ee33018f9ftargetxGsha256-d4c6f5fffcd9231ce88d5bb4089e52ca46af47f180146af58c6a0b5ee33018f9mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-5e0fcdd2b50c81c59af41332c732e441c897a5e84833b3c1436dd6320c29ca19dkindldocument.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-b636499925ad5d1f56f96bab8cfe2f97edc6f48ef063ee26fa53b44a70c4efe7jprovenance¡hobjectIdxGsha256-b636499925ad5d1f56f96bab8cfe2f97edc6f48ef063ee26fa53b44a70c4efe7ftargetxGsha256-b636499925ad5d1f56f96bab8cfe2f97edc6f48ef063ee26fa53b44a70c4efe7mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-619b96c1b24720bf6df70d68b337475389840c4bc352fff87f696d292adf642bdkindldocument.putgparents€gpayloadxGsha256-1269f143ca81e88d69a3cda8c503118d2cad496399b04bb33cd1ba3ef9e4d77ajprovenance¡hobjectIdxGsha256-1269f143ca81e88d69a3cda8c503118d2cad496399b04bb33cd1ba3ef9e4d77aftargetxGsha256-1269f143ca81e88d69a3cda8c503118d2cad496399b04bb33cd1ba3ef9e4d77amtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-6395c75b4a6099feb0bf433bfe0a1a311d0d780a2751c267c3317cc799f0a919dkindldocument.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-bd59e9ebb72d5be196557cc0801621c19650f35d88918d8ea80e105b6ca1c4b2jprovenance¡hobjectIdxGsha256-bd59e9ebb72d5be196557cc0801621c19650f35d88918d8ea80e105b6ca1c4b2ftargetxGsha256-bd59e9ebb72d5be196557cc0801621c19650f35d88918d8ea80e105b6ca1c4b2mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-68d7d88fd97fb46ff8fd46e0661c43b2f9524a4b68ec084469e8d18d3ff69a21dkindjsource.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-fd28669e76998f27f651dcd2f1fa212be13f03ba25216a2014df4560d4299199jprovenance¡hobjectIdxGsha256-fd28669e76998f27f651dcd2f1fa212be13f03ba25216a2014df4560d4299199ftargetxGsha256-fd28669e76998f27f651dcd2f1fa212be13f03ba25216a2014df4560d4299199mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-8ad8ebba44d98623252cf630eaa0fb4d02348bfe4b00987b83aa9ad08a6c71e3dkindldocument.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-d499a8116f750bf7376e2a9478dc1796b8ace5ac28ccf7c0b33b1cf1993e23e3jprovenance¡hobjectIdxGsha256-d499a8116f750bf7376e2a9478dc1796b8ace5ac28ccf7c0b33b1cf1993e23e3ftargetxGsha256-d499a8116f750bf7376e2a9478dc1796b8ace5ac28ccf7c0b33b1cf1993e23e3mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-8b0fcaac4b201ee8fa590263bcffd16eb8278cca197ac442c005ec14caaa9cdadkindldocument.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-d987f42c7c59dbb3e558d5d848fa31c4f64066b36941bd4aec2b6ff6cfc8f53djprovenance¡hobjectIdxGsha256-d987f42c7c59dbb3e558d5d848fa31c4f64066b36941bd4aec2b6ff6cfc8f53dftargetxGsha256-d987f42c7c59dbb3e558d5d848fa31c4f64066b36941bd4aec2b6ff6cfc8f53dmtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-9247baaf2abf4a338d12551be117d92f0490b482aeb26fb5716f790e0359dca5dkindldocument.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-68bbf7f3d5738b01e2d6620b1e944287987fd9cefcb5b79c1fbffb1785a09cbejprovenance¡hobjectIdxGsha256-68bbf7f3d5738b01e2d6620b1e944287987fd9cefcb5b79c1fbffb1785a09cbeftargetxGsha256-68bbf7f3d5738b01e2d6620b1e944287987fd9cefcb5b79c1fbffb1785a09cbemtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-939faf12f7daec5f1695b5b6132fd5247d98098bb333cbb47c0742106fef66dcdkindldocument.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-b3227935b5e3b03b12407fa3d0223b0a6d6e1839c6882b05b9c6eb07e34882cfjprovenance¡hobjectIdxGsha256-b3227935b5e3b03b12407fa3d0223b0a6d6e1839c6882b05b9c6eb07e34882cfftargetxGsha256-b3227935b5e3b03b12407fa3d0223b0a6d6e1839c6882b05b9c6eb07e34882cfmtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-9944420d60c7c3f20d4a0d519189e6c00230b16d4b3944ffa390a52cb5144a67dkindjsource.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-68b4c62932cb920ef5928dc7ba7abbe90eb1ca0c50e9ba90f166becc4492fe4djprovenance¡hobjectIdxGsha256-68b4c62932cb920ef5928dc7ba7abbe90eb1ca0c50e9ba90f166becc4492fe4dftargetxGsha256-68b4c62932cb920ef5928dc7ba7abbe90eb1ca0c50e9ba90f166becc4492fe4dmtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounter
bidxGsha256-b1ff391cf2acdc8c5a005f716e3dd21df5fa14ad34155437ba425f7773f977e3dkindjsource.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-6851502b2d1f262e32cf009ceb24d5bb9d42231531f185bf0fc8c7e1bd5e5080jprovenance¡hobjectIdxGsha256-6851502b2d1f262e32cf009ceb24d5bb9d42231531f185bf0fc8c7e1bd5e5080ftargetxGsha256-6851502b2d1f262e32cf009ceb24d5bb9d42231531f185bf0fc8c7e1bd5e5080mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-b969fbb164d48bd6fcc14c9c1a69189ece87b4504d778070113c03c1f85e0febdkindjsource.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-af8cf48f258264ed133d4facec028e4316f2b1a5f6d67cdcd6a6faa7e551a9d1jprovenance¡hobjectIdxGsha256-af8cf48f258264ed133d4facec028e4316f2b1a5f6d67cdcd6a6faa7e551a9d1ftargetxGsha256-af8cf48f258264ed133d4facec028e4316f2b1a5f6d67cdcd6a6faa7e551a9d1mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-c6a8ab2007ff1835304d6df2238d0674850ef80a7a4de6c1001e51144d75007bdkindjsource.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-bb2b61bc7b0ee5132ed1fe147d1d48f8f9289ab1cdb40e13b82a09b59cda3ab2jprovenance¡hobjectIdxGsha256-bb2b61bc7b0ee5132ed1fe147d1d48f8f9289ab1cdb40e13b82a09b59cda3ab2ftargetxGsha256-bb2b61bc7b0ee5132ed1fe147d1d48f8f9289ab1cdb40e13b82a09b59cda3ab2mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-cc481f2b68f448427398a2037dd8e3d032806a016c57ba4d0cbe74a37a5b13f2dkindjsource.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-199e68f52fe6267863033be1fbf410457f595de26301c16945cd39ea9c3cc5bcjprovenance¡hobjectIdxGsha256-199e68f52fe6267863033be1fbf410457f595de26301c16945cd39ea9c3cc5bcftargetxGsha256-199e68f52fe6267863033be1fbf410457f595de26301c16945cd39ea9c3cc5bcmtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-d32063ecbe6686aac9a93cd3aef1e8de5778e3b95ca146f8ab584f71ffed2e5adkindldocument.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-44c81fe219407e78439cadc5452a71277de39c16cda04133678fb511b96831ddjprovenance¡hobjectIdxGsha256-44c81fe219407e78439cadc5452a71277de39c16cda04133678fb511b96831ddftargetxGsha256-44c81fe219407e78439cadc5452a71277de39c16cda04133678fb511b96831ddmtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-d3b40def5399a0e549b489c1ad3b7f96cd967515310332399ded82896c009f15dkindldocument.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-f4095d3c583e7461badeed5bc8658f24e8c5f8d3ff645b70be11028a5fdb54ecjprovenance¡hobjectIdxGsha256-f4095d3c583e7461badeed5bc8658f24e8c5f8d3ff645b70be11028a5fdb54ecftargetxGsha256-f4095d3c583e7461badeed5bc8658f24e8c5f8d3ff645b70be11028a5fdb54ecmtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-e5282bc51ffb87cacf515d71474d93e67cb4ff7222d32bc162ae7cc103ce567bdkindjsource.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-f2e928e124d7ded8825a963728e80fffe3a9c733fb43bcc861098bea268f8719jprovenance¡hobjectIdxGsha256-f2e928e124d7ded8825a963728e80fffe3a9c733fb43bcc861098bea268f8719ftargetxGsha256-f2e928e124d7ded8825a963728e80fffe3a9c733fb43bcc861098bea268f8719mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-f1e5a062a337ef3efbcb0d7a763904443918694f35e61c805616610e78b54870dkindldocument.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-9685aa28c4ddb7f261b49c614a03f70e0509b4069dab1ebda3e6f820d1c2d889jprovenance¡hobjectIdxGsha256-9685aa28c4ddb7f261b49c614a03f70e0509b4069dab1ebda3e6f820d1c2d889ftargetxGsha256-9685aa28c4ddb7f261b49c614a03f70e0509b4069dab1ebda3e6f820d1c2d889mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-f934406fbd48ce33c9262a8ddc719aa5be734f441a57eba004bae1167dcffaf0dkindldocument.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-7579fd41df4c7a3e793bc04e281a0f2f1fbb31798f3676336bb2c0b8b0fbf369jprovenance¡hobjectIdxGsha256-7579fd41df4c7a3e793bc04e281a0f2f1fbb31798f3676336bb2c0b8b0fbf369ftargetxGsha256-7579fd41df4c7a3e793bc04e281a0f2f1fbb31798f3676336bb2c0b8b0fbf369mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionªeactorqknolo-v4-migratorlactorCounterbidxGsha256-fd2686f5a6dfc3eefa5ea9b01c26d771bb074226742f308d79692adfe70bf923dkindjsource.putgparentsxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gpayloadxGsha256-8dd696b33bd97df0e4bedf1ab201b5f8e3f2e2043dc8ec4b98206ac2311bfe02jprovenance¡hobjectIdxGsha256-8dd696b33bd97df0e4bedf1ab201b5f8e3f2e2043dc8ec4b98206ac2311bfe02ftargetxGsha256-8dd696b33bd97df0e4bedf1ab201b5f8e3f2e2043dc8ec4b98206ac2311bfe02mtransactionIdxGsha256-b64f407fb8c97e650b00c2da5039e4f30d2022525317b556b01451eea811e8b3gversionKSEG  G      |£pÀe`|—}V|0ÂvÖŽ´¿ç ÊH~Zi­eactorqknolo-v4-migratorieventRootxGsha256-a0810e706f11d1da6766869b6b2e9d873a05123e93f20b178ffcbf2e420e0364reventSegmentDigestxGsha256-1f81eebd5bdc6628a9cd9ecced8dd9802634158fb38384985b259aaec012789ajobjectRootxGsha256-3d43cb59dfa17104fbf1e3026a6b6a081ce8114aa877fc58d50b4e43d2a7cae1sobjectSegmentDigestxGsha256-132b5b100a966bc03b483c430a9f84767673f908bdc8450eed8fea9f1b13f1d9gparents€jpolicyRootxGsha256-92a33041498984ee8303a0862158a6624776418fd5b854fb293d66e63febee9foruntimeContractxGsha256-21c14cbf9139b7cb3186e9081668668a7dab762f490a11c4a86bed5c0ead2ca8jschemaRootxGsha256-babaf64c1c0e673411504b9d015f62e723b9058035b7d5e5678bbb30163a5e86hsequenceotransactionRootxGsha256-a8c6a206a994e6d0944efbed1ba346daa0a04a61d729fb5b32cf55e360884eb1gversioneviews¡glexicalxGsha256-7a5f6cbb0254b12485a7c9cec2b79a5d84783e73b1fe6ae5647c180c8c5888dc