Mora / Datasets / Orders / Lab 703857

Order from Lab 703857

Code review

Pull requests from private companies where a person reviewed the change, the author revised it in response, and the result after merge is known (clean, reverted, hotfixed, or traced to an incident). Each one must come with per-push diffs that apply on their base commit, line-anchored reviewer comments, and outcome links that can be checked against the git history.

How answering works

Any number of people fill one order. The lab receives one dataset, in its own columns.

  1. Your agent reads the order

    It looks at what you hold and says which columns you can fill, and what is missing.

  2. Mora checks what you send

    Every record: no copies, no personal data, not already public, not on Mora from someone else.

  3. The lab pays per record it accepts

    The lab named the price. You fill as much of the order as you can.

The columns the lab wants

You do not need every column. Mora says which ones each dataset fills.

ColumnTypeWhat it must hold
pr_id, repo_id, base_sha, head_sha, merge_shaid / textThe pull request and the exact commits it was opened against and merged as, so the diff can be replayed on the repository
title, description, author_id, opened_at, merged, merged_attext / id / date / booleanWhat the author said the change does, who wrote it (as a stable id), and whether and when it merged
revisions: revision_n, head_sha, pushed_at, diffnumber / text / date / textOne row per push: the diff against base at that moment, so the change before and after review can be compared
review_comments: comment_id, thread_id, in_reply_to, reviewer_id, revision_n, paid / text / number / dateEach reviewer comment anchored to a file and line of a given revision, with its thread
resolved, addressed_by_revision, is_botboolean / number / booleanWhether the comment was resolved, which revision answered it, and whether a bot wrote it
reviews: review_id, reviewer_id, state, revision_n, submitted_atid / label / number / dateThe decision on each review: approved, changes requested, or commented
outcomes: outcome, revert_sha, fix_pr_id, incident_id, severity, detected_at, lilabel / text / id / dateWhat happened after merge, and where the link between the change and the incident comes from (postmortem, revert commit, blame)
codegit bundleThe repository history, so base_sha checks out and diffs can be applied

How much, in the lab's words

200,000 merged pull requests with at least one human inline comment followed by a revision, from 40 companies or more (none above 10%), of which at least 3,000 have an outcome other than clean, plus a slice merged after our training cutoff for evaluation

  • pull-requests
  • revisions
  • review-comments
  • reviews
  • ci
  • outcomes
  • code