Evaluate dsh-superagent-prototype in a Test Repo: Agent-Assisted Setup and a First Ticket

Introduction

A proactive coding agent that watches your repository, takes work from issues, and answers with pull requests sounds like a shortcut to a whole extra team member. dsh-superagent-prototype, published by the DeepSeek team on October 31, 2026, is exactly that kind of agent, but it is not a product you deploy and forget. This tutorial evaluates it the way its own documentation asks you to: in a throwaway repository, through the agent-assisted install in SETUP.md, and with one small ticket from an issue comment to a pull request.

Quick answer

Evaluate dsh-superagent-prototype only in a test repository: ask a coding agent to run the SETUP.md install with you, approve one plan, then trigger one small ticket and review its pull request yourself.

Evidence

Start with the numbers, because they set the right level of trust. As of November 1, 2026, the repository at deepseek-ai/dsh-superagent-prototype had 3,658 stars and 338 forks, but only 1 commit and 0 releases, and it was created on October 31, 2026. That combination, thousands of stars on a one-day-old, single-commit repository, is a signal of intense curiosity, not of maturity. The license is MIT, so you can read, copy, and adapt the code, but the age and commit count mean details can change without warning.

The most important sentence in the whole project is its opening disclaimer, quoted verbatim from the README: “Research prototype for reference only, not an official DeepSeek product.” The same paragraph adds that the code grew out of the team’s day-to-day development work, that much of the engineering was done by coding agents, and that it may still contain bugs with inconsistent style and practices. It also warns that the published version is not identical to the team’s internal version, and that some designs in the accompanying article are directions for future development, not implemented in this prototype. Every step below respects that warning.

The design idea is worth understanding before you install anything. The README describes dsh-superagent as a proactive coding agent for one GitHub repository, packaged as a bundle for DeepSeek Harness. It listens to GitHub and, optionally, Feishu. When someone allowed asks it for work, it opens a work item, works in its own agent session and directory, and answers in the original conversation, for example with a pull request. A web panel shows what needs a person. The guiding principle is to take as little human attention as possible while giving as much value as possible.

Prerequisites

You need a machine you can install developer tools on, plus accounts you control. The documented prerequisites are Node.js 22.19 or later (or 24 or later), pnpm, git 2.32 or later, tmux, Python 3.8 or later, GitHub CLI 2.48 or later, and DSH within the engines.dsh range in the repository’s package.json. DSH also needs a model provider configured. You also need a GitHub account for yourself as the operator, a second dedicated GitHub account for the agent, and a throwaway GitHub repository that an experimental agent can write to. Feishu is optional, and this tutorial uses GitHub only, which is the recommended starting point. Finally, you need a coding agent of your own to run the assisted setup with you.

Step-by-step guide

Step 1: Clone the prototype and start the assisted setup

Clone the prototype repository to your machine, then ask your coding agent to read the setup file and set it up with you. The README gives the exact request, and you should use it rather than paraphrasing it, so the agent follows the documented script:

Read SETUP.md in this repository and set up dsh-superagent with me.

The full setup procedure lives in SETUP.md. Read it yourself before you start, so you can spot it when the agent skips a step or asks for something the file says it should not ask for, such as a secret in the chat.

Step 2: Let the agent run its probe first

Before the first question, the agent reads only SETUP.md and the philosophy document, then runs one probe command from the prototype checkout. The probe reports the versions of Node, pnpm, git, tmux, Python, the GitHub CLI, and DSH, checks who you are signed in as on GitHub, and checks whether a dsh-superagent profile already exists. A missing tool becomes an item of the plan, not a question. If a profile already exists, the agent should ask whether to continue it and keep its configuration, following the upgrade notes in the deployment document. Do not let the agent jump ahead to questions before the probe result is in, because the probe answers some of them, including your GitHub login, which becomes the recommended allowed list later.

Step 3: Answer the nine questions, one at a time

The setup then asks nine questions in the order work flows through the system: first where requests come from, then who may ask for work, then what the executor does with a ticket. Each question comes as its own short message with a recommended answer you can accept with one word, plus one sentence on why it matters. For an evaluation, the answers are straightforward. Take requests from GitHub only, and point the agent at your throwaway repository. Use a dedicated agent account, never your own account, and create it by hand if it does not exist yet. On GitHub, allow only your own login at first; a short allowed list keeps noise low while you learn how the agent behaves. For the checkout, accept the recommended new clone under ~/src/<name>, and let the agent’s commits name the agent account as author, so its work is visibly its own. For auto-merge, accept the recommended answer, which is no: people merge until the agent’s pull requests have been right for a while.

Do not be drawn into extra questions about values that have good defaults. The setup lists those in the plan instead: a profile named superagent, kept apart from the profile you work in; port 3082; a state directory at <home>/.dsh/state/superagent; an hourly check that repairs dsh-superagent when it is not running; a branch prefix of agent/; and an accept mode of observe, which records proactive work without doing it. The accept mode matters for an evaluation: observe is the safe starting point for a prototype.

Step 4: Approve one full plan, then do the operator steps by hand

Once every answer is collected, the agent sends one message with the full plan, under 300 words, covering what will be installed, the accounts and CLIs, the settings to save, the operator steps, and the defaults. It executes only after you confirm that plan. This is the checkpoint where the prototype’s philosophy shows up in practice: the agent collects everything before acting, rather than installing piece by piece while it is still asking questions.

The operator steps come first, and they are yours alone. Following the accounts section of the deployment document, you create the agent account, accept the repository invitation as the agent account, and run the sign-in command for each command line wrapper. Secrets never go into the chat; steps that need your hands go into one message with the exact commands to run in your own terminal. After you report that the operator steps are done, the agent checks each login itself before continuing. If a login check fails, fix that account before letting the install proceed.

Step 5: Install, start, and save the settings through the management API

With the operator steps done, the agent follows the install, machine configuration, and start and stop sections of the deployment document. It skips cloning the prototype repository itself, since you are already in its checkout, and clones your test repository if you asked for a new clone. If you are continuing an existing profile, it must skip the command that would replace the profile, because that command would delete the profile’s cordis.patch.yml file, and it edits that file instead.

Settings are saved through the management API in a fixed order: the github Source first, the feishu Source if used, the policy block with the allowed lists, and the executor block with a prepare command from your checkout answer. Prompts are saved last, with the commit identity and auto-merge answer folded into the delivery requirements. A save that returns restarting: true restarts dsh-superagent, and the agent waits until its status command no longer reports a problem before the next request. This ordering is not cosmetic: the policy block is what stops strangers from driving your agent, so it must be in place before the agent starts listening.

Step 6: Verify the status and run one small ticket

Verification is a single command: scripts/dsh-superagent status must exit 0 with no setup problem. If you run in your own DSH profile, the setup also creates the hourly check that repairs the agent when it is not running; otherwise the agent hands you the prompt to schedule that check yourself. The handover message then gives you the panel URL and tells you that the Superagent entry in the sidebar is where to look, plus the three operator rules: start narrow, tune for less noise, and own what the agent says.

Now run the evaluation ticket. From an account on the allowed list, comment @<agent login> <request> on an issue in your test repository, with a small, self-contained request such as fixing a typo in a documentation file. A ticket should appear on the overview within about a minute. Review the resulting pull request line by line before merging it. The branch name should carry the agent/ prefix, and the commits should name the agent account. That review is the evaluation: a prototype earns a real repository only when its first few pull requests are right, and the default of no auto-merge exists precisely so that you, not the agent, make that call.

Step 7: Decide what the prototype earned

After the first ticket, judge the prototype on three things. First, did it respect its boundaries: only the allowed account could drive it, it worked in its own directory, and it answered in the original conversation? Second, did it respect your attention: did it speak only when you had to act or could use a result, and did the panel show a clean list of what needed a person? Third, was the pull request actually good: correct, minimal, and in the style of the repository? If any answer is no, narrow the scope further, adjust the prompts and rules, and try another small ticket, not widen access. If all three answers are yes across several tickets, you can consider a second repository.

Troubleshooting

The agent asks for a secret in the chat. Stop and refuse. The setup rules say never to ask for a secret in the chat. Steps that need your hands belong in one message with the exact commands to run in your own terminal. If the agent asks for a token or secret directly, it has left the documented script, and you should point it back to SETUP.md before continuing.

A settings save is refused. Read the refusal carefully, because a refused save names the setup problem in its response body. Find that problem in the troubleshooting table of the deployment document, fix it, and continue. Do not retry the same save unchanged. Most refused saves trace back to an operator step that was skipped or a login that was never verified.

The status command keeps failing after a restart. A save that returns restarting: true restarts dsh-superagent, and the next request must wait until scripts/dsh-superagent status no longer exits 1. If the status still fails after waiting, the restart has surfaced a setup problem rather than a transient delay. Re-read the settings from the management API, compare them against the plan you approved, and work through the deployment document’s troubleshooting table.

The agent acts for the wrong person, or on the wrong repository. Check the policy block first. The agent acts only for people on the allowed lists, and the recommended starting list is only your own login. If the allowed list is correct, check the github Source settings for the repository it is watching.

FAQ

Is dsh-superagent an official DeepSeek product?

No. The README opens by calling it a research prototype for reference only, not an official DeepSeek product, and warns that it is not production-ready software. Treat every result as an evaluation, and review its code and pull requests before trusting either.

Can I install dsh-superagent with a single command?

No. The README says setup has many steps, so an agent does it with you. You ask your coding agent to read SETUP.md and set it up with you, answer one short question at a time, approve one full plan, and only then let it install and configure everything.

What does the agent actually do once it is running?

It watches one GitHub repository, and optionally Feishu chats. When someone on the allowed list asks it for work, it opens a work item, works in its own agent session and directory, and answers in the original conversation, for example with a pull request. A web panel shows only what needs a person.

Should the agent use my own GitHub account?

No. The setup guide recommends a dedicated agent account, never the operator’s own account, because everything the agent writes appears under that name. Creating that account is one of the operator steps you do by hand.

What is the safest first ticket to give it?

A small, self-contained change in a throwaway repository, such as fixing a typo in documentation or adding a short test. The guide’s operator rules are start narrow, tune for less noise, and own what the agent says, so a tiny first ticket is the right way to learn how it behaves.

Conclusion

dsh-superagent-prototype is an interesting evaluation, not a deployment. Its philosophy, that an agent should take as little human attention as possible and speak only when someone must act or can use a result, is sound, and its setup script enforces it with a short allowed list, a dedicated agent account, no auto-merge, and an observe mode for proactive work. But the repository is one day old with a single commit, its own README says it is research code and not an official DeepSeek product, and its published form differs from both the internal version and the design article. Run it in a throwaway repository, review every pull request, and let it earn a real repository over several small tickets. If you want a contrasting skill that disciplines how a change is written and tested, read the ub-mode skill tutorial, and for a skills bundle covering the pull request loop end to end, read the tstack agent skills tutorial. For agent automation inside workflows, the n8n complete guide is the next place to look.

Frequently asked questions