Skip to content

Benchmark PR 5 - #8

Open
celmis-codereviewer wants to merge 32 commits into
cr-base-5from
cr-pr-5
Open

celmis-codereviewer wants to merge 32 commits into
cr-base-5from
cr-pr-5

Conversation

@celmis-codereviewer

Copy link
Copy Markdown

Benchmark reproduction of ai-code-review-evaluation#5

armenzg and others added 30 commits June 20, 2025 12:49
…o 'low' (#93927)"

This reverts commit 8d04522.

Co-authored-by: roaga <47861399+roaga@users.noreply.github.com>
Missed in the initial commit, leading to some relevant logs being
unannotated.
We have had a few tasks get killed at 10% rollout.
Also add a test, so that this doesn't happen again
Fixes DE-129 and DE-156

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
These transitions should be matching
…` (#93946)

Use `project_id` on the replay record instead of the URL (where it does
not always exist).

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: getsantry[bot] <66042841+getsantry[bot]@users.noreply.github.com>
Also fixed `replay.view_html` -> `replay.view-html`

---------

Co-authored-by: Michelle Zhang <56095982+michellewzhang@users.noreply.github.com>
…948)

gets `npx @typescript/native-preview` passing again
The conditions associated with a DCG can change over time, and it's good
if we can be completely confident that they're consistent within a given
task execution.
This is unused and most regex experiments have required broader changes
to ensure that regexes are evaluated in a specific order (ex:
traceparent). Removing this for now to simplify the code and very
slightly improve runtime performance.
From some testing (on feedback lists of all different lengths), this
prompt seems to work better. It doesn't write overly long sentences and
also does a better job at "summarizing" versus just mentioning a few
specific topics and leaving out others.
Just remove a couple custom Flex* classes in favor of the Flex primitive
This has been killed a few times.

Refs SENTRY-42M7
…n table (#93892)

<!-- Describe your PR here. -->

[ticket](https://linear.app/getsentry/issue/ID-156/grouping-info-remove-type-field-from-ui)
The Type field in the Grouping Info section of the issue details page
was redundant.
This removes the Type row from all variant types while keeping the
underlying data structure intact.

before
![Screenshot 2025-06-20 at 12 00
54 PM](https://github.com/user-attachments/assets/97ca72da-0a52-4446-9825-cd4fcb505adf)

after
![Screenshot 2025-06-20 at 11 59
29 AM](https://github.com/user-attachments/assets/a4284d2b-c9f5-442f-b010-7fe72a598e39)
### Changes
Related to this PR: getsentry/sentry#93810. This
is part 1 of the change, which is pulling out the new component and just
adding it to the repo. Also includes some simplification of the logic in
the base component.

Part 2 will be replacing tables in widgets.

### Before/After

There is no UI change as the table is not being used yet. There is a new
story page for the component.
…93943)

to prevent this issue from becoming too noisy, add a noise config
Unfortunately, 'event_data' went from being the variable for current
event context to being the complete parsed data from Redis, and we
continued logging it per group.
That's more data than we should be logging even arguably once, let alone
per group.
Co-authored-by: Abdullah Khan <abdullahkhan@PG9Y57YDXQ.local>
Adds some simple analytics to our endpoint so we can begin building a
dashboard in Amplitude.
Previously, explore supported multiple y axis per chart, so each
visualize supported multiple y axis. That functionality has since been
removed for simplicity so update the types here to match. Keep in mind
that saved queries still store them as an array so when
serializing/deserializing, we still need to treat it as an array.
We'll need the `useGetTraceItemAttributeKeys` hook in other places so
refactoring it so that it can exported.
mrduncan and others added 2 commits June 20, 2025 13:20
When the max segment ID is null the process fails. We should exit early
since if there aren't any segments to delete there's nothing to do.

@celmis-codereviewer celmis-codereviewer left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVED — no blocking findings

Full findings and scope are in the review summary comment on this pull request — one persistent comment, updated in place on every run.

celmis-codereviewer

This comment was marked as outdated.

celmis-codereviewer

This comment was marked as outdated.

celmis-codereviewer

This comment was marked as outdated.

celmis-codereviewer

This comment was marked as outdated.

@celmis-codereviewer celmis-codereviewer left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CHANGES REQUESTED — blocking findings

Full findings and scope are in the review summary comment on this pull request — one persistent comment, updated in place on every run.


@staticmethod
def get_merged_pr_single_issue_template(title: str, url: str, environment: str) -> str:
truncated_title = PRCommentWorkflow._truncate_title(title)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why: CommitContextIntegration._truncate_title is called via PRCommentWorkflow._truncate_title on line 601, but PRCommentWorkflow does not define _truncate_title, which causes an AttributeError when get_merged_pr_single_issue_template is executed.

🟠 AttributeError calling non-existent method on PRCommentWorkflow

In CommitContextIntegration.get_merged_pr_single_issue_template, _truncate_title is called via PRCommentWorkflow._truncate_title(title). However, _truncate_title is defined on CommitContextIntegration, not PRCommentWorkflow. Invoking PRCommentWorkflow._truncate_title will raise an AttributeError at runtime.

Call self._truncate_title(title) or CommitContextIntegration._truncate_title(title) instead.

Suggested change
truncated_title = PRCommentWorkflow._truncate_title(title)
truncated_title = self._truncate_title(title)

agent: defect · rule: defect.attribute-error · confidence: 0.95

title=data.get("title", ""),
timestamp=data.get("timestamp", 0.0),
message=data.get("message", ""),
)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why: nodestore.backend.get_multi returns a dictionary keyed by node ID, so using zip(error_ids, events.values()) on line 117 pairs error IDs with event data positionally, resulting in wrong event IDs mapped to event payloads if any key is missing or reordered.

🟠 Mismatched error details due to positional zip with nodestore dict values

nodestore.backend.get_multi(node_ids) returns a dictionary of {node_id: data} containing only found entries. Zipping error_ids with events.values() assumes events.values() aligns positionally with error_ids. If any event ID is missing from nodestore, events.values() will have fewer elements, causing subsequent error_ids to be paired with the wrong event payload data.

Suggested change
)
return [
ErrorEvent(
category="error",
id=event_id,
title=data.get("title", ""),
timestamp=data.get("timestamp", 0.0),
message=data.get("message", ""),
)
for event_id, node_id in zip(error_ids, node_ids)
if (data := events.get(node_id)) is not None
]

agent: defect · rule: defect.wrong-data-mapping · confidence: 0.95

def update(self, instance: Detector, validated_data: dict[str, Any]):
instance.name = validated_data.get("name", instance.name)
instance.type = validated_data.get("detector_type", instance.group_type).slug

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why: BaseDetectorTypeValidator registers the detector type under key 'type', so validated_data.get('detector_type', ...) on line 65 always returns the fallback instance.group_type, silently preventing detector type updates.

🟠 Detector type update is silently ignored due to wrong key lookup in validated_data

In BaseDetectorTypeValidator, the serializer field is defined as type = serializers.CharField(). When validated, DRF populates validated_data["type"] (as used in create() on line 133). In update(), the code queries validated_data.get("detector_type", instance.group_type). Because the key in validated_data is "type", validated_data.get("detector_type") is always None, causing update() to constantly fall back to instance.group_type and ignore the updated type value.

Suggested change
if "type" in validated_data:
instance.type = validated_data["type"].slug

agent: defect · rule: defect.dict-key-mismatch · confidence: 0.95

timestamp = serializers.IntegerField(required=False, min_value=0)

def validate_timestamp(self, value: int) -> int:
"""Validate that age is absent, but timestamp is present."""

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why: When age is 0 in the request payload, self.initial_data.get('age') on line 51 evaluates to falsy 0, allowing both age and timestamp to be accepted simultaneously without raising a validation error.

🟡 Falsy check on age allows simultaneous timestamp and age when age is 0

self.initial_data.get("age") returns 0 when age: 0 is passed in the report payload. Because 0 is falsy in Python, if self.initial_data.get("age") evaluates to False, bypassing the validation check that requires age to be absent when timestamp is present. Checking key membership with "age" in self.initial_data ensures the condition triggers correctly even when age is 0.

Suggested change
"""Validate that age is absent, but timestamp is present."""
if "age" in self.initial_data:
raise serializers.ValidationError("If timestamp is present, age must be absent")

agent: defect · rule: defect.falsy-check · confidence: 0.90

@celmis-codereviewer

Copy link
Copy Markdown
Author

🤖 Code Review for PR #8

⚙ ADJUSTED — graph context partial (85 of 100 changed files): 15 of 100 changed files have no symbols in the index; 7 of them are still in the checkout the index was built from (src/sentry/options/defaults.py, static/app/views/dashboards/widgets/common/types.tsx, static/app/views/explore/contexts/pageParamsContext/index.spec.tsx, static/app/views/explore/contexts/pageParamsContext/visualizes.spec.tsx, static/app/views/explore/toolbar/index.spec.tsx and 2 more) — the index is stale there, or the extractor could not parse it; run analyzer generate or index it from the Repositories page (POST /api/repos/index-all); 8 of them are not in that checkout at all (static/app/components/codecov/datePicker/dateSelector.tsx, static/app/components/codecov/repoPicker/repoSelector.tsx, static/app/views/explore/charts/index.tsx, static/app/views/explore/hooks/useTraceItemAttributeValues.spec.tsx, static/app/views/explore/hooks/useTraceItemAttributeValues.tsx and 3 more) — this PR's base is older than the indexed revision, so those files were renamed or deleted before it and no re-index can bring them back; there is nothing to fix.

CHANGES REQUESTED — blocking findings

Findings

  • 🟠 Error: 3
  • 🟡 Warning: 1

Scope

  • Files changed: 100
  • Lines: +1824 / -969

Performance

  • Analysis time: 1534.3s · agents: cve, structural, contract, security, defect · tokens: 86,891/30,150

Powered by Code Analyzer · context: tree-sitter graph + cve, structural, contract, security, defect

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.