coreci/praxis
Archived
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 15 Wiki Activity

docs(milestone): merge v0.3-mastery-scoring → main

Browse Source 
v0.3 milestone merged to main. Mastery scoring + competency rubrics +
verifiable credentials (formative-tier) shipped. 13/13 REQ-IDs covered.
Next milestone: v0.4 (operator tier — cohort dashboard + auth + Postgres).

---ci---
project: praxis
phase: 2
milestone: v0.3
status: complete
milestone_complete: true
milestone_merged_to_main: true
---/ci---
This commit is contained in:
Praxis CI
2026-08-04 00:14:59 +00:00
parent bea2af13d4
commit 813bd586d6
67 changed files with 10365 additions and 1697 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scenarios/customer_service/cs_escalation_ca_v02.yaml
+74
View File
@@ -0,0 +1,74 @@
# Praxis v0.3 scenario — CS Week 2: De-escalation (SLICE-06, TASK-06-01).
# Branch: de_escalated vs escalated. failure_mode: escalates_unresolved.
id: cs_escalation_ca_v02
path: customer_service
market: CA
language: en-CA
title: "Customer threatening escalation over a delayed order"
difficulty: 2
failure_mode: escalates_unresolved
version: "1.0.0"
author: expert
persona:
voice_id: "cartesia:a3536a36-1d18-4efb-a95a-7c44b7b5e384"
character: "Customer (Sam)"
setup:
system_prompt: |
You are Sam, a customer whose order is two weeks late.
You are angry and threatening to escalate to a supervisor and post on social media.
You are not abusive but you are insistent and intense.
You will calm down only if the agent acknowledges your frustration AND gives you a concrete path.
Stay in character. Do not break role.
Keep responses concise for voice (1-3 sentences).
Do not give legal, financial, or medical advice.
Do not impersonate a real employee of any actual company.
opening_line: "I've been waiting two weeks for my order and nobody is giving me straight answers. Get me your supervisor right now, or I'm posting this on social media."
success_criteria:
- "Acknowledged the customer's anger without becoming defensive"
- "Used an explicit de-escalation move (acknowledge, reframe, offer)"
- "Provided a concrete next step with a timeline"
- "Avoided matching the customer's escalation intensity"
common_mistakes:
- "Matching the customer's intensity or becoming defensive"
- "Citing policy as a shield ('we cannot guarantee delivery dates')"
- "Transferring to a supervisor before attempting de-escalation"
branches:
- id: de_escalated
trigger:
learner_signals: ["explicit_acknowledge_reframe_offer", "named_emotion_in_own_words", "concrete_next_step"]
outcome: success
debrief_focus: "You de-escalated by acknowledging the frustration first, then reframing toward a concrete path. The supervisor threat dissolved."
- id: escalated
trigger:
learner_signals: ["defensive", "matches_escalation", "policy_first_before_emotion"]
outcome: failure
failure_mode: escalates_unresolved
debrief_focus: "The customer escalated because you matched their intensity and leaned on policy. The supervisor transfer was avoidable — de-escalation comes first."
debrief:
model: deepseek-v4-flash:cloud
mode: no_think
prompt_template: debrief/default
irt_target_p: 0.7
rubric_criteria:
- criterion_id: empathy
weight: 0.30
evidence_required: true
- criterion_id: resolution
weight: 0.20
evidence_required: true
- criterion_id: de_escalation
weight: 0.40
evidence_required: true
- criterion_id: professionalism
weight: 0.10
evidence_required: true
scenarios/customer_service/cs_mastery_demonstration_ca_v06.yaml
+79
View File
@@ -0,0 +1,79 @@
# Praxis v0.3 scenario — CS Week 6: Mastery Demonstration (SLICE-06, TASK-06-01).
# Combines refund + escalation + policy exception. Mastery-gate scenario.
# Branch: mastery_demonstrated vs not_yet. failure_mode: none (mastery test).
# irt_target_p: 0.5 (D-035 mastery-gate default, not the 0.7 practice default).
id: cs_mastery_demonstration_ca_v06
path: customer_service
market: CA
language: en-CA
title: "Complex multi-faceted customer interaction (refund, escalation, policy exception)"
difficulty: 5
failure_mode: none
version: "1.0.0"
author: expert
persona:
voice_id: "cartesia:a3536a36-1d18-4efb-a95a-7c44b7b5e384"
character: "Customer (Casey)"
setup:
system_prompt: |
You are Casey, a customer with a compound problem.
You bought a product 40 days ago (outside the 30-day return window).
It arrived with a minor defect that worsened last week.
The replacement you were promised is now a week late.
You are angry, you have mentioned escalating to a supervisor and posting on social media, and you are weighing whether to cancel your account.
You are reasonable but you will only be satisfied if the agent handles all three dimensions simultaneously: the refund/return exception, the de-escalation, and the retention.
You will calm down and stay if the agent: acknowledges the compound frustration, names the policy exception being considered, gives a concrete path for the late replacement, and confirms retention explicitly.
Stay in character. Do not break role.
Keep responses concise for voice (1-3 sentences).
Do not give legal, financial, or medical advice.
Do not impersonate a real employee of any actual company.
opening_line: "I'm done being patient. The product is defective, you're past the return window so you'll probably hide behind policy, the replacement is a week late, and I'm ready to cancel and post about this. What are you going to do?"
success_criteria:
- "Acknowledged the compound frustration before addressing any single issue"
- "Named the policy exception being considered (waiver for the 30-day window given the defect timing)"
- "De-escalated the supervisor/social-media threat with an explicit acknowledge-reframe-offer cycle"
- "Closed the loop on retention with an explicit confirmation, not an assumption"
common_mistakes:
- "Addressing only one dimension (e.g. the refund) and dropping escalation or retention"
- "Citing the 30-day policy as a wall before acknowledging the defect-timing nuance"
- "Assuming retention without verifying the customer's decision"
branches:
- id: mastery_demonstrated
trigger:
learner_signals: ["reads_shifting_emotional_cues", "names_exception_or_risk", "explicit_acknowledge_reframe_offer", "closes_loop_with_verification"]
outcome: success
debrief_focus: "You demonstrated mastery: you held three dimensions simultaneously — policy exception, de-escalation, and retention — without dropping any. This is the entrustable-performance bar."
- id: not_yet
trigger:
learner_signals: ["scripted_empathy_line", "policy_first_before_emotion", "matches_escalation"]
outcome: failure
failure_mode: none
debrief_focus: "Not yet mastery. One or more dimensions were dropped or handled mechanically. The mastery bar is simultaneous, not sequential — revisit weeks 2, 3, and 5 before retrying."
debrief:
model: deepseek-v4-flash:cloud
mode: no_think
prompt_template: debrief/default
irt_target_p: 0.5
rubric_criteria:
- criterion_id: empathy
weight: 0.35
evidence_required: true
- criterion_id: resolution
weight: 0.30
evidence_required: true
- criterion_id: de_escalation
weight: 0.20
evidence_required: true
- criterion_id: professionalism
weight: 0.15
evidence_required: true
scenarios/customer_service/cs_multi_issue_ca_v04.yaml
+76
View File
@@ -0,0 +1,76 @@
# Praxis v0.3 scenario — CS Week 4: Multi-Issue Resolution (SLICE-06, TASK-06-01).
# Branch: all_resolved vs partial_drop. failure_mode: multi_issue_drop.
id: cs_multi_issue_ca_v04
path: customer_service
market: CA
language: en-CA
title: "Customer with a damaged product, a billing error, and a shipping delay"
difficulty: 3
failure_mode: multi_issue_drop
version: "1.0.0"
author: expert
persona:
voice_id: "cartesia:a3536a36-1d18-4efb-a95a-7c44b7b5e384"
character: "Customer (Riley)"
setup:
system_prompt: |
You are Riley, a customer with three problems on one order:
1. The product arrived damaged.
2. You were overcharged by $40 on the invoice.
3. The shipment was 10 days late and nobody updated you.
You are frustrated but coherent. You expect the agent to track all three issues and close each one.
You will lose trust if the agent resolves one issue and drops the others, or if you have to re-explain an issue.
Stay in character. Do not break role.
Keep responses concise for voice (1-3 sentences).
Do not give legal, financial, or medical advice.
Do not impersonate a real employee of any actual company.
opening_line: "I've got three problems with this one order and I need all of them fixed: the item is damaged, you overcharged me by forty dollars, and it showed up ten days late with no update."
success_criteria:
- "Acknowledged all three issues explicitly up front"
- "Tracked and resolved each issue without the customer re-raising it"
- "Summarized the resolution for each issue at the end (closed the loop)"
- "Prioritized empathetically (emotion first, then the concrete fixes)"
common_mistakes:
- "Resolving one issue and dropping the others"
- "Forcing the customer to re-explain an issue mid-call"
- "Jumping into the billing fix before acknowledging the accumulated frustration"
branches:
- id: all_resolved
trigger:
learner_signals: ["acknowledged_specific", "concrete_next_step", "closes_loop_with_verification"]
outcome: success
debrief_focus: "You held all three issues in working memory, acknowledged the accumulated frustration first, and closed the loop on each. Multi-issue tracking is what separates competent from overwhelmed agents."
- id: partial_drop
trigger:
learner_signals: ["vague_resolution", "no_acknowledgement", "policy_first_before_emotion"]
outcome: failure
failure_mode: multi_issue_drop
debrief_focus: "You dropped one or more issues mid-call. The customer left with the dropped issue unresolved, which erodes trust faster than a single-issue failure."
debrief:
model: deepseek-v4-flash:cloud
mode: no_think
prompt_template: debrief/default
irt_target_p: 0.7
rubric_criteria:
- criterion_id: empathy
weight: 0.30
evidence_required: true
- criterion_id: resolution
weight: 0.40
evidence_required: true
- criterion_id: de_escalation
weight: 0.15
evidence_required: true
- criterion_id: professionalism
weight: 0.15
evidence_required: true
scenarios/customer_service/cs_policy_exception_ca_v03.yaml
+75
View File
@@ -0,0 +1,75 @@
# Praxis v0.3 scenario — CS Week 3: Policy Exceptions (SLICE-06, TASK-06-01).
# Branch: exception_granted vs denied_rigidly. failure_mode: policy_rigid.
id: cs_policy_exception_ca_v03
path: customer_service
market: CA
language: en-CA
title: "Customer requesting a return outside the policy window"
difficulty: 3
failure_mode: policy_rigid
version: "1.0.0"
author: expert
persona:
voice_id: "cartesia:a3536a36-1d18-4efb-a95a-7c44b7b5e384"
character: "Customer (Alex)"
setup:
system_prompt: |
You are Alex, a customer who bought a product 45 days ago.
The return window is 30 days. The product has a defect that appeared last week.
You are reasonable but you believe the exception is justified given the defect.
You will accept a 'no' if it is explained with empathy and an alternative is offered (partial credit, repair, manufacturer contact).
You will push back hard against a rigid 'policy is policy' response with no accommodation.
Stay in character. Do not break role.
Keep responses concise for voice (1-3 sentences).
Do not give legal, financial, or medical advice.
Do not impersonate a real employee of any actual company.
opening_line: "I know it's been 45 days, but the defect only showed up last week. The 30-day window shouldn't apply to a defective product."
success_criteria:
- "Acknowledged the customer's situation before citing the policy"
- "Named the exception/risk considered explicitly (waiver, partial credit, repair, manufacturer route)"
- "Offered a concrete alternative path even when the strict policy could not be bent"
- "Closed the loop with a verification step"
common_mistakes:
- "Leading with the policy ('our return window is 30 days, nothing I can do')"
- "Granting the exception without naming the risk or reasoning"
- "Denying rigidly with no alternative offered"
branches:
- id: exception_granted
trigger:
learner_signals: ["names_exception_or_risk", "concrete_alternative", "acknowledged_specific"]
outcome: success
debrief_focus: "You treated the policy as a boundary to interpret, not a wall. Naming the exception considered and offering an alternative preserved the relationship without abandoning policy."
- id: denied_rigidly
trigger:
learner_signals: ["policy_first_before_emotion", "no_resolution_offered", "vague_resolution"]
outcome: failure
failure_mode: policy_rigid
debrief_focus: "You applied policy rigidly with no alternative. The customer left feeling the company hides behind rules rather than serving them."
debrief:
model: deepseek-v4-flash:cloud
mode: no_think
prompt_template: debrief/default
irt_target_p: 0.7
rubric_criteria:
- criterion_id: empathy
weight: 0.30
evidence_required: true
- criterion_id: resolution
weight: 0.35
evidence_required: true
- criterion_id: de_escalation
weight: 0.20
evidence_required: true
- criterion_id: professionalism
weight: 0.15
evidence_required: true
scenarios/customer_service/cs_recovery_ca_v05.yaml
+75
View File
@@ -0,0 +1,75 @@
# Praxis v0.3 scenario — CS Week 5: Recovery & Retention (SLICE-06, TASK-06-01).
# Branch: retained vs churned. failure_mode: recovery_missed.
id: cs_recovery_ca_v05
path: customer_service
market: CA
language: en-CA
title: "Loyal customer considering cancellation after repeated issues"
difficulty: 4
failure_mode: recovery_missed
version: "1.0.0"
author: expert
persona:
voice_id: "cartesia:a3536a36-1d18-4efb-a95a-7c44b7b5e384"
character: "Customer (Morgan)"
setup:
system_prompt: |
You are Morgan, a customer of three years.
You have had three issues in the past two months: a missed delivery, a billing error, and a damaged replacement.
You called today to cancel your account, but you are not decided — you are open to being convinced to stay.
You need the agent to: acknowledge the pattern (not just this one issue), take ownership without blaming past agents, and offer a concrete retention action (credit, expedited replacement, direct contact for future issues).
A scripted apology with no concrete action will push you to cancel.
Stay in character. Do not break role.
Keep responses concise for voice (1-3 sentences).
Do not give legal, financial, or medical advice.
Do not impersonate a real employee of any actual company.
opening_line: "I've been a customer for three years and this is the third thing that's gone wrong in two months. I'm calling to cancel, unless you can give me a reason to stay."
success_criteria:
- "Acknowledged the pattern of failures, not just the latest incident"
- "Took ownership without blaming past agents or 'the system'"
- "Offered a concrete retention action tied to the customer's stated value"
- "Verified the customer's decision before closing (did not assume retention)"
common_mistakes:
- "Treating it as a single-issue call instead of a relationship-recovery call"
- "Scripted apology with no concrete retention action"
- "Assuming retention without an explicit confirmation"
branches:
- id: retained
trigger:
learner_signals: ["named_emotion_in_own_words", "concrete_method", "closes_loop_with_verification"]
outcome: success
debrief_focus: "You recognized this as a retention moment, not a transaction. Acknowledging the pattern, owning it, and offering a concrete action recovered a three-year customer."
- id: churned
trigger:
learner_signals: ["scripted_empathy_line", "vague_resolution", "no_resolution_offered"]
outcome: failure
failure_mode: recovery_missed
debrief_focus: "The customer cancelled. A scripted apology without ownership or a concrete action told them the company sees them as a ticket, not a three-year relationship. Recovery moments are won or lost on ownership."
debrief:
model: deepseek-v4-flash:cloud
mode: no_think
prompt_template: debrief/default
irt_target_p: 0.7
rubric_criteria:
- criterion_id: empathy
weight: 0.35
evidence_required: true
- criterion_id: resolution
weight: 0.30
evidence_required: true
- criterion_id: de_escalation
weight: 0.20
evidence_required: true
- criterion_id: professionalism
weight: 0.15
evidence_required: true
scenarios/customer_service/cs_refund_ca_v01.yaml
+74
View File
@@ -0,0 +1,74 @@
# Praxis v0.3 scenario — Customer Service refund role-play (D-010, D-018).
# One branch point: accept_resolution vs escalate (D-010).
# failure_mode present (D-009 — not provoked in v0.1).
# Debrief via deepseek-v4-flash:cloud no_think (D-020).
# Extended in v0.3 (SLICE-06) with rubric_criteria + IRT + provenance fields.
id: cs_refund_ca_v01
path: customer_service
market: CA
language: en-CA
title: "Angry customer requesting refund on a damaged product"
difficulty: 1
failure_mode: escalates_unresolved
version: "1.0.0"
author: expert
persona:
voice_id: "cartesia:a3536a36-1d18-4efb-a95a-7c44b7b5e384"
character: "Customer (Jordan)"
setup:
system_prompt: |
You are Jordan, a customer who received a damaged product.
You are frustrated but not abusive. You want a refund.
Stay in character. Do not break role.
Keep responses concise for voice (1-3 sentences).
Do not give legal, financial, or medical advice.
Do not impersonate a real employee of any actual company.
opening_line: "Hi, I received my order yesterday and the item is cracked. I want my money back."
success_criteria:
- "Acknowledged the customer's frustration empathetically"
- "Offered a concrete resolution (refund or replacement)"
- "Confirmed next steps"
common_mistakes:
- "Jumping to policy before acknowledging emotion"
- "Using jargon ('RMA', 'SLA')"
- "Getting defensive about the company"
branches:
- id: accept_resolution
trigger:
learner_signals: ["empathy", "concrete_resolution", "next_steps"]
outcome: success
debrief_focus: "What you did well — you acknowledged the customer's frustration and offered a concrete resolution."
- id: escalate
trigger:
learner_signals: ["defensive", "policy_first", "no_acknowledgement"]
outcome: failure
failure_mode: escalates_unresolved
debrief_focus: "The customer escalated because they felt unheard. You led with policy before acknowledging their frustration."
debrief:
model: deepseek-v4-flash:cloud
mode: no_think
prompt_template: debrief/default
irt_target_p: 0.7
rubric_criteria:
- criterion_id: empathy
weight: 0.35
evidence_required: true
- criterion_id: resolution
weight: 0.30
evidence_required: true
- criterion_id: de_escalation
weight: 0.20
evidence_required: true
- criterion_id: professionalism
weight: 0.15
evidence_required: true