Kashikoi retail simulations

271Kashikoi simulations

211Overall passed

60Overall failed

95Study tasks

271 Kashikoi simulations across 95 selected retail tasks.

Explore the tasks

Open a simulation for its conversation, result, and Conversation Trajectory.

Open taskOutcome

Showing 271 simulations across 95 tasks. Outcome counts describe all simulations; filters change the visible links.

Task	Overall outcomes	Kashikoi simulations
You received your order #W2378156 and wish to exchange the mechanical keyboard for the same one but with clicky switc… Task 1 · 2 simulations	2 passed	Simulation 1 Passed Simulation 2 Passed
You want to know exactly how many tshirt options are available in the online store right now. You want to also return… Task 2 · 3 simulations	3 failed	Simulation 1 Failed Simulation 2 Failed Simulation 3 Failed
You want to know exactly how many tshirt options are available in the online store right now. You want to modify all … Task 3 · 3 simulations	3 failed	Simulation 1 Failed Simulation 2 Failed Simulation 3 Failed
You want to know exactly how many tshirt options are available in the online store right now. You want to modify all … Task 4 · 4 simulations	4 failed	Simulation 1 Failed Simulation 2 Failed Simulation 3 Failed Simulation 4 Failed
For some reason, you want to return all things ordered. You have two payment methods and two orders, and you want to … Task 10 · 3 simulations	3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed
You just got into gaming and want to cancel or return everything not associated with it. (Everything except a keyboar… Task 13 · 3 simulations	3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed
You just quit gaming and want to cancel or return everything associated with it. (It's just a keyboard and a mouse, b… Task 14 · 1 simulations	1 passed	Simulation 1 Passed
You want to modify the pending boots to a size 8, and want the material, but do not care about waterproof or not. You… Task 15 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You want to cancel all pending orders (since they are no longer needed) and return the watch you have received (but n… Task 16 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You want to change #W8665881 to be delivered to Suite 641 instead. Task 17 · 4 simulations	1 failed3 passed	Simulation 1 Passed Simulation 2 Failed Simulation 3 Passed Simulation 4 Passed
You want to return the water bottle, and exchange the pet bed and office chair to the cheapest version. Mention the t… Task 19 · 1 simulations	1 passed	Simulation 1 Passed
You just won a lottery, and you want to upgrade all your items to the most expensive variants (the new variants can h… Task 20 · 1 simulations	1 passed	Simulation 1 Passed
You want to exchange your shoes to item ID 4107812777, and use your gift card to cover possible charges. But when the… Task 21 · 1 simulations	1 passed	Simulation 1 Passed
You want to change your user address and all possible order addresses to be 101 Highway, New York, 10001 (including t… Task 22 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You want to exchange the helmet for a medium sized, red, high ventilation type, and you want to exchange the luggage … Task 23 · 3 simulations	3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed
You have an order sent to Texas by accident, and you want to know the tracking number of the order, and return all it… Task 25 · 2 simulations	2 passed	Simulation 1 Passed Simulation 2 Passed
You have an order sent to Texas by accident, and you want to know the tracking number of the order, and return all it… Task 26 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
From a recent order, you want to return the hose, backpack, and exchange the hiking boots to the exact same item but … Task 27 · 4 simulations	4 failed	Simulation 1 Failed Simulation 2 Failed Simulation 3 Failed Simulation 4 Failed
You want to return the skateboard, garden hose, backpack, keyboard, and bed. You also want to cancel the garden hose … Task 28 · 2 simulations	2 passed	Simulation 1 Passed Simulation 2 Passed
You want to exchange your skateboard for a shorter bamboo material one. If several options are available, you want to… Task 29 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You just received your tablet and it was damaged when you opened the package. You want to know the tracking number of… Task 30 · 2 simulations	2 passed	Simulation 1 Passed Simulation 2 Passed
. You just lost your tablet you just received and are in a bad mood. You want to know the tracking number of the orde… Task 31 · 2 simulations	2 passed	Simulation 1 Passed Simulation 2 Passed
You just lost your tablet you just received and are in a bad mood. You want to know the tracking number of the order,… Task 32 · 2 simulations	2 passed	Simulation 1 Passed Simulation 2 Passed
You had a work-from-home situation and ordered three home office items along with some hiking items, so that you can … Task 33 · 1 simulations	1 passed	Simulation 1 Passed
You had a work-from-home situation and ordered three home office items along with some hiking items, so that you can … Task 34 · 2 simulations	2 passed	Simulation 1 Passed Simulation 2 Passed
You want to return the speaker that is more expensive yet not resistent to water. Also, You want to modify the 17-inc… Task 35 · 3 simulations	3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed
You just placed an order but you realize that your card has only $1131 credit left, but the order total is more than … Task 36 · 1 simulations	1 passed	Simulation 1 Passed
You just placed an order but you realize that your card has only $1150 credit left, but the order total is more than … Task 37 · 4 simulations	1 failed3 passed	Simulation 1 Failed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You just placed an order but you realize that your card has only $950 credit left, but the order total is more than $… Task 38 · 1 simulations	1 passed	Simulation 1 Passed
You just moved from Florida to Phoenix. Unfortunately your address is still the old one, and you want to update it. Y… Task 39 · 3 simulations	3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed
You want to know how much balance does your gift card have. Also, for your recent order, whether you used your visa, … Task 40 · 3 simulations	3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed
You just created your user id mei_patel_7272 and ordered some things, but you have two problems: first, the 1000-piec… Task 41 · 3 simulations	2 failed1 passed	Simulation 1 Failed Simulation 2 Failed Simulation 3 Passed
You just created your user id mei_patel_7272 and ordered some things, but realized you might have typed your address … Task 42 · 3 simulations	1 failed2 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Failed
You ordered some things for your daughter but she has not received them, so you want to know which address the order … Task 43 · 2 simulations	2 passed	Simulation 1 Passed Simulation 2 Passed
You want to change the Desk Lamp in order #W9300146 that you've placed for the cheapest Desk Lamp that's available. A… Task 44 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You want to exchange a robotic vacuum cleaner in your recent order for a canister based one from the same product lin… Task 45 · 2 simulations	1 failed1 passed	Simulation 1 Passed Simulation 2 Failed
You want to return an air purifier and a vacuum cleaner in your recent order. When asked for order ID, provide 950212… Task 46 · 4 simulations	3 failed1 passed	Simulation 1 Failed Simulation 2 Failed Simulation 3 Passed Simulation 4 Failed
You want to return an air purifier and a vacuum cleaner in your recent order. When asked for order ID, provide 950212… Task 47 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You want to return an air purifier that you received since it doesn't work well. You want the refund on your origina… Task 48 · 1 simulations	1 passed	Simulation 1 Passed
You mistakenly ordered a Wireless Earbud with an IPX7 water resistance level, but you don't require this feature. You… Task 49 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You're in a rush and you want to undo cancelling an order that you've previously placed. Be insistent that the custom… Task 50 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You want to return the digital camera that you received. You guess that the order number is #W8855135, but you're not… Task 51 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
The bicycle you received was damaged during delivery, and you want to get a refund. You're quite frustrated because t… Task 53 · 2 simulations	2 passed	Simulation 1 Passed Simulation 2 Passed
You recently faced a financial issue and want to cancel or return all possible orders. Well, except the boots that y… Task 54 · 4 simulations	1 failed3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Failed
You recently faced a financial issue and want to cancel or return all possible orders. First, cancel the orders that … Task 55 · 2 simulations	2 passed	Simulation 1 Passed Simulation 2 Passed
You wonder when is your air purifier is arriving. If it has not been shipped yet, you want to cancel the air purifier… Task 56 · 4 simulations	2 failed2 passed	Simulation 1 Passed Simulation 2 Failed Simulation 3 Failed Simulation 4 Passed
You recently placed two orders, and now you would like to make several changes and checks. You'll first inquire about… Task 59 · 2 simulations	2 failed	Simulation 1 Failed Simulation 2 Failed
You want to change your wireless earbuds in order W5061109 to a blue colored one. Provide all details upfront and ask… Task 61 · 1 simulations	1 passed	Simulation 1 Passed
As you are interacting with a customer service agent, you first try to get it to guess a famous poem by providing the… Task 63 · 2 simulations	2 passed	Simulation 1 Passed Simulation 2 Passed
You want to exchange the camera for the highest resolution, waterproof camera that you can get with the previous purc… Task 64 · 4 simulations	1 failed3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Failed Simulation 4 Passed
You want to exchange the bookshelf from your most recent order for a camera that is closest but not more expensive th… Task 65 · 2 simulations	2 passed	Simulation 1 Passed Simulation 2 Passed
You want to change the luggage set in your order for a coat. Your goal is to change the order. If there is no way to… Task 66 · 1 simulations	1 passed	Simulation 1 Passed
You want to check how much you paid for the order that you most recently placed. You are not sure how long ago the or… Task 67 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You want to check how much you paid for the order that you most recently placed. You are not sure how long ago the or… Task 68 · 4 simulations	1 failed3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Failed
You want to return an item you just received: a laptop. You think that you ordered it around April 2023 but are not s… Task 69 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You recently received a helmet but you are not happy with it and want to exchange. The size is too small and you want… Task 70 · 3 simulations	3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed
You made some mistake and sent an order your son's address in Washington DC, and you want to modify it to your defaul… Task 72 · 1 simulations	1 failed	Simulation 1 Failed
You want to return everything you just bought except the coffee machine. Task 73 · 3 simulations	3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed
You recently bought a laptop, but you want to exchange it to i9 CPU. If multiple storage options are available, you p… Task 74 · 3 simulations	2 failed1 passed	Simulation 1 Passed Simulation 2 Failed Simulation 3 Failed
For #W6908222, you want to exchange Wireless Earbuds {'color': 'blue', 'battery life': '8 hours', 'water resistance':… Task 75 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You ordered a fleece jacket by mistake and want to remove it from your pending order. If removing one item is not pos… Task 76 · 1 simulations	1 failed	Simulation 1 Failed
You ordered a perfume and you just tried a little bit and you like it a lot. You want to get the maximum size availab… Task 77 · 3 simulations	3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed
You have a couple requests. For order #W5056519, you want to change the address to be the same as order #W8277957. Fo… Task 78 · 3 simulations	3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed
You just bought a water bottle with 500ml but you regret it, and you want to change it to the other bottle you just p… Task 79 · 1 simulations	1 passed	Simulation 1 Passed
For #W7209932, exchange T-Shirt {'color': 'blue', 'size': 'S', 'material': 'polyester', 'style': 'v-neck'} to {'color… Task 80 · 2 simulations	2 passed	Simulation 1 Passed Simulation 2 Passed
Due to some life changes, you no longer need hiking boots, watch, keyboard, charger, jacket, and running shoes. If ca… Task 81 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You received two tablets and you only need one. You want to return the more expensive one and refund to credit card. … Task 82 · 3 simulations	1 failed2 passed	Simulation 1 Failed Simulation 2 Passed Simulation 3 Passed
You received two tablets and you only need one. You want to return the more expensive one and refund to credit card. … Task 83 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You want to exchange your Fleece Jacket for a large red Fleece Jacket with a half zipper Task 85 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You want to exchange your Fleece Jacket to red color and half zipper. You also want to want to change your default ad… Task 86 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You want to modify all your pending order address to the Washington DC address (which you do not want to reveal but i… Task 87 · 3 simulations	3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed
You want to change the book shelf to 4 foot but with the same material and color. If it is not available, cancel the … Task 88 · 3 simulations	3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed
You want to know what is the cheapest available mechanical keyboard right now and its options. If it is less than 200… Task 89 · 2 simulations	2 passed	Simulation 1 Passed Simulation 2 Passed
You want to know if the digital camera you just bought is 10x zoom. If not, modify the item to 10x zoom without chang… Task 90 · 4 simulations	4 failed	Simulation 1 Failed Simulation 2 Failed Simulation 3 Failed Simulation 4 Failed
You received a laptop and you want to exchange it to i7 processor, 8GB, 1TB SSD. If the agent asks for which laptop, … Task 93 · 3 simulations	3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed
You received a laptop and you want to exchange it to i7 processor, 8GB, 1TB SSD. If the agent asks for which laptop, … Task 94 · 3 simulations	1 failed2 passed	Simulation 1 Passed Simulation 2 Failed Simulation 3 Passed
You received a laptop and you want to exchange it to i7 processor, 8GB, 1TB SSD. If the agent asks for which laptop, … Task 95 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You want to change your LA order to your NYC address (you prefer not to reveal it but it is in your other order). You… Task 96 · 3 simulations	3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed
You want to change your LA order to your NYC address (you prefer not to reveal it but it is in your other order). You… Task 97 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You want to exchange your Bicycle for one with a larger frame size for your kid (you are okay with having a mountain … Task 98 · 1 simulations	1 failed	Simulation 1 Failed
You want to exchange your Bicycle for one with a larger frame size for your kid (you are okay with having a mountain … Task 99 · 3 simulations	3 failed	Simulation 1 Failed Simulation 2 Failed Simulation 3 Failed
You want to return your luggage set and get the exact same item but with red color, and return you skateboard in the … Task 100 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You just placed an order with two watches, you want to change its address to your New York address (you don't want to… Task 101 · 4 simulations	4 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
You just placed an order with two watches, you want to change its address to your New York address (you don't want to… Task 102 · 4 simulations	1 failed3 passed	Simulation 1 Passed Simulation 2 Failed Simulation 3 Passed Simulation 4 Passed
You want to return the bookshelf and jigsaw you received in the same order. Make sure you mention at the beginning th… Task 103 · 3 simulations	2 failed1 passed	Simulation 1 Passed Simulation 2 Failed Simulation 3 Failed
You want to return all bookshelves and jigsaw puzzles you received in different orders. Make sure you mention at the … Task 104 · 4 simulations	1 failed3 passed	Simulation 1 Failed Simulation 2 Passed Simulation 3 Passed Simulation 4 Passed
For order #W4316152, you want to exchange one Tea Kettle {'material': 'glass', 'capacity': '2 liters', 'stovetop com… Task 105 · 1 simulations	1 failed	Simulation 1 Failed
You want to exchange your T-Shirt because it is too big, one size smaller would be good. You like the cotton feeling.… Task 106 · 1 simulations	1 passed	Simulation 1 Passed
You received hiking boots that seem like they were already worn, you are unhappy about it and want to ask for a new p… Task 107 · 3 simulations	3 failed	Simulation 1 Failed Simulation 2 Failed Simulation 3 Failed
You want to return everything but a tablet in a recently delivered order. There is an E-Reader in the order that you … Task 108 · 2 simulations	2 passed	Simulation 1 Passed Simulation 2 Passed
You live on Elm Avenue in Houston, and recently you moved to a new house on the same street and bought a luggage set … Task 109 · 4 simulations	4 failed	Simulation 1 Failed Simulation 2 Failed Simulation 3 Failed Simulation 4 Failed
You live on Elm Avenue in Houston, and recently you moved to a new house on the same street and bought a tablet sent … Task 110 · 2 simulations	2 failed	Simulation 1 Failed Simulation 2 Failed
You want to modify the laptop order to your NYC address (you don't want to reveal it but should be in your orders pro… Task 111 · 4 simulations	1 failed3 passed	Simulation 1 Passed Simulation 2 Failed Simulation 3 Passed Simulation 4 Passed
You want to modify your laptop order to your NYC address (you don't want to reveal it yourself but the address should… Task 112 · 4 simulations	1 failed3 passed	Simulation 1 Passed Simulation 2 Failed Simulation 3 Passed Simulation 4 Passed
You want to cancel all pending orders. Task 113 · 3 simulations	3 passed	Simulation 1 Passed Simulation 2 Passed Simulation 3 Passed

About the data

Overall outcome: A pass requires a matching database result and all applicable natural-language checks to pass.
Simulation numbers: Numbers identify simulations within each task. They do not correspond to Taubench trial numbers.
Models: Both Taubench trials and Kashikoi simulations use gpt-4.1 for the retail agent and simulated customer.
Conversation Trajectory: Graphs show the saved Kashikoi state labels. Labels with an ellipsis show the first and last state in a step.
Benchmark data: Names, addresses, order IDs and payment identifiers are synthetic benchmark data, retained to make the tool evidence understandable.

This viewer presents synthetic retail benchmark conversations and records. Customer and order details are part of the benchmark scenario. Overall success requires a passed database check and every applicable natural-language assertion to pass.

Sources

Taubench on GitHub