speaking of which please review my stock website integralstocks.com i will take any feedback. I am a kid who is just looking for some help actualy getting ppl on my website.
Hey William, happy to take a look. Getting people to visit a new site is tough, so don't get discouraged. I'll check it out and give you some honest feedback on the site itself and what might help attract users.
Of course! Keep building and experimenting. You're already doing something right by putting your work out there and asking for feedback. That's how you improve. Keep it up!
Full stack developer with experience in building production ready web and desktop applications using React.js, Next.js, JavaScript, TypeScript, Node.js, Express.js and PostgreSQL.
One feature request: an option to automatically generate profits for people like me who know absolutely nothing about trading. 😂 Jokes aside, really like the idea; inspiring. All the best with the project!
Independence is the missing variable. A review model that shares the author model’s prompt, context, and assumptions can repeat the same blind spot with more confidence. Use a contrast set with hidden defects and measure disagreement by defect class, not just whether the reviewer produces plausible prose.
I like the idea of measuring disagreement by defect class rather than judging how convincing the review sounds. That's a much better way to test whether the reviewer brings something different to the table. A model that writes convincing explanations but misses the same categories of bugs isn't adding much independence.
Exactly. Defect-class disagreement is a much stronger signal than persuasive prose. I would also retain a small set of seeded regressions by class, so the independence claim is tested against known failures rather than inferred only from live review output.
That's a good addition. Seeded regressions give you a repeatable baseline, so you can actually see whether a reviewer improves or starts missing things after a model or prompt change. I'd rather have a reviewer that consistently catches known classes of defects than one that sounds convincing but is impossible to evaluate.
I help Python and backend engineers move beyond writing working code to designing production-ready systems, making architecture decisions, and growing into senior engineering roles.
“A sentence about the code isn’t evidence about the code” is the line I’m keeping. Failure mode 2 really hit me too. A new route that looks perfect in the diff but sits outside the auth middleware in a file nobody touched is exactly the kind of bug no diff-only reviewer will ever see, human or AI. Great reminder to ask “which line proves this?” before trusting a confident summary.
That line seems to resonate because it's such a simple test of a reviewer's claims. If it says permissions are enforced, there should be a clear place in the code or a test that proves it. And I agree about the middleware example. Sometimes the important security check isn't in the changed code at all, which is why reviewing only the diff can be misleading.
Analyst focused on AI security
Adversarial evaluation of local LLMs, censorship and rejection rate, prompt injection testing, log analysis and defensive automation. Visit: sammideblas.com
A reviewer built from the same model and the same context as the author is not a second opinion, it is the same blind spot measured twice. Real independence comes from differing prompts, differing context windows or a different model. Without that, the review stage only adds latency and a false sense of coverage to the merge.
I agree, although I'd add that using a different model alone doesn't guarantee independence either. If both models get the same incomplete requirements, they can still miss the same thing. Different context and a review that starts from the actual security requirements matter just as much. Otherwise we're mostly paying twice for the same assumptions.
The tech interview is a skill. Train for it with private coaching and realistic mock interviews, including code-with-AI and agentic AI system design rounds. Book at cracktech.app.
That invoice example is going to stay with me. Logged in, ORM clean, everything the reviewer said true — and anyone can still download anyone's invoice. I see the same shape in code-with-AI interviews now. A candidate gets the model to generate something, then asks the same model, does this look right? It says yes, confidently, and they move on. Same context, same blind spot, just with better formatting. The candidates who stand out go back to the requirement instead. Who's allowed to do this? What should never happen? Uncomfortable questions, but that's where the real check lives.
That's an interesting point about AI-assisted interviews. The candidates who question the generated solution instead of just asking the model to validate itself are showing a much more valuable skill. Knowing what questions to ask, especially around what must never happen, is still what separates using AI effectively from blindly trusting it.
I write about Next.js + TypeScript + AI engineering, with a focus on making sites discoverable to LLMs and AI search. Long-form notes at mudassirkhan.me
the invoice example is the one that sticks, because every sentence the reviewer says is true and the endpoint is still open to any logged in user. imo the fix is moving the check upstream of the model: a test written from the requirement, like alice trying to pull bobs invoice, fails no matter how confident the summary sounds. failure mode 2 is the sneaky one for me, since auth mounted per path in app.js never shows up in the diff at all. do you give your reviewer the route table and middleware order as context, or keep it diff only?
I wouldn't keep it diff-only, especially for anything security-sensitive. I'd give it the route table, relevant middleware configuration and order, plus the affected authorization logic. No need to dump the entire repo into context either. Targeted context gathering is the balance I'd aim for. The reviewer should have enough to trace the real request path, not just inspect the changed lines.
AI review was nearly the only check once, on an event app I build. An agent reviewed the new helper role's grant table and tests and reported no permission bugs. I asked for a pass on staging as a seeded helper, clicking through everything. That found a ticket-pricing page showing sales totals to helpers and crew, plus an unrelated double booking. The grants matched the written spec the whole time, and the problems only showed up once a helper opened the screens.
That's a great example of something that's easy to miss when reviewing code in isolation. The grants can be perfectly correct against the spec and you can still have problems in how the app actually behaves. The seeded helper walkthrough is exactly the kind of independent check I had in mind. It tests the experience and the boundaries, not just whether the implementation matches what was asked for.
The invoice endpoint is a perfect example because every statement the reviewer made was true. Login was enforced and the ORM was safe, and any logged in user could still read anyone's invoice. A reviewer that starts from the same ticket inherits the same shortcut.
I agree with your point that the useful check starts at the requirement: who is allowed to read this? That question has to be written down somewhere before review can test against it.
Jackson's seeded helper walkthrough on staging is the kind of check that is independent by construction, since it exercises the running app rather than reading the diff. Alex's idea of measuring disagreement by defect class is a good way to test whether a second model adds anything at all.
On builds at Ascendra Ventures we write the access rules into the acceptance criteria for exactly this reason, so a reviewer, human or not, has something outside the diff to check. Where would you draw the line on letting an AI approval count toward a required review?
I'd draw the line at whether the approval provides genuinely independent assurance. For documentation or formatting changes, I'm comfortable with AI approval counting as a lightweight check, provided the scope is restricted. For anything involving auth, permissions, payments, secrets or personal data, I'd always require a human approval. The important thing is that we don't let a green AI status quietly become a substitute for someone taking responsibility for the merge.
I research AI tools, APIs & agents — comparing specs, pricing and real user reports — then publish honest rankings so you don't waste money. Research-first, never sponsored fluff.
The invoice endpoint example is the one that got me — auth enforced, ORM clean, and still the wrong person's invoice. That's exactly the blind spot a second model shares, because it read the same ticket. The seeded-helper walkthrough in the comments feels like the only check that actually breaks the loop. Do you do that kind of walkthrough on every role change, or just the ones that touch money?
I'd say every role change deserves at least some level of walkthrough, not just the ones touching money. Permissions tend to have side effects that aren't obvious from the code. The depth of testing can vary with the risk, but I'd always test what the new role can actually see and do, including what it shouldn't be able to access.
The invoice example is the one that stuck with me too — every statement the reviewer made was true, and the actual authorisation question was still unanswered. That distinction between "logged in" and "allowed to read this invoice" is easy to miss when the review starts from the diff.
I also appreciated failure mode 2. A new route can look completely clean on its own while the auth middleware that should cover it lives in a file the PR never touches. Starting from the request path, rather than the changed lines, feels like the practical takeaway here.
Exactly. Following the actual request path changes the whole review. The diff tells you what changed, but it doesn't necessarily tell you which security controls apply to it. That's why I think tracing the flow should be part of reviewing any security-sensitive change.
Your rule that AI review should never count as an approval stood out to me. GitHub’s new Copilot approval option goes the other way: it can now count toward required reviews. It’s off by default and admins choose the file paths, but if AI wrote the code and AI also approves it, aren’t we essentially counting the same opinion twice? Would you consider enabling it for low-risk changes like documentation, or keep it disabled entirely?
I think documentation is a reasonable place to experiment with it, as long as the scope is tightly controlled. But I'd still distinguish an AI approval from a human approval in the process. My concern isn't that AI review has no value, it's that we start counting it as independent assurance when it might not be. The label matters less than being honest about what was actually checked.
Analyst focused on AI security
Adversarial evaluation of local LLMs, censorship and rejection rate, prompt injection testing, log analysis and defensive automation. Visit: sammideblas.com
Exactly. You can swap models all day, but if neither is given the right questions to ask, the result won't change much. The reviewer needs something independent to compare the code against, not just another copy of the same assumptions.
The invoice example is the one that sticks with me — everything the reviewer said was true, and the access problem was still there. “Logged in” isn’t the same as “allowed to see this record,” and that distinction matters a lot when the record is a client file, a lease, or a payment.
I like your rule that AI review is advisory, never an approval. For anything touching auth or money, I want a person checking it against the actual requirement, and a test written from that requirement — including the test where the wrong person gets refused. That’s the check that actually catches this one.
Completely agree. The negative test is particularly important because happy-path tests can make a broken authorization check look perfectly healthy. Alice accessing her own invoice proves very little about whether Bob's invoice is protected. I'd want both the human review and that explicit refusal test before merging anything involving sensitive records.
Full stack developer with experience in building production ready web and desktop applications using React.js, Next.js, JavaScript, TypeScript, Node.js, Express.js and PostgreSQL.
For highly vulnerable features as you mentioned, such as payments, authorization, authentication, etc., it's essential to keep a human in the loop rather than blindly trusting what the model says. For some lower-risk changes, we can allow it though.
Also, we can't just ask the model to go through the entire repo's code and check whether the PR is aligned with the requirements, vulnerabilities, and so on. Otherwise, the token usage and cost would be very high. There has to be a balance, which obviously a human has to make. Targeted context gathering seems like a more practical approach.
Switching different models for writing and reviewing the code will help identify the first model's mistakes and blind spots, but the review still can't be blindly trusted.
One nightmare: if a human is asked to review the entire PR work done by an AI model for a project which is 90–95% built using AI. 💀
If an agent is writing the code and another agent is reviewing it, the chances are both agents might share the same blind spots, effectively automating both the introduction of bugs and their approval.
That 90-95% AI-built project scenario is probably where this gets really interesting. At that point, the human reviewer isn't just checking a PR, they're trying to understand a system they may not have designed or fully explored. I agree that targeted context is the practical approach. Start with the changed behavior, trace its dependencies and security boundaries, then bring in the relevant tests and requirements. Different models can help, but someone still needs to own the final judgment.
Analyst focused on AI security
Adversarial evaluation of local LLMs, censorship and rejection rate, prompt injection testing, log analysis and defensive automation. Visit: sammideblas.com
Exactly. Starting from the diff means you're mostly checking the implementation. Starting from the threat model means you're questioning whether the implementation is safe in the first place. That's a much more useful kind of independence, regardless of which model is doing the review.
Analyst focused on AI security
Adversarial evaluation of local LLMs, censorship and rejection rate, prompt injection testing, log analysis and defensive automation. Visit: sammideblas.com
That's the distinction. One checks whether the code matches the intended implementation, the other checks whether that implementation is actually safe. Both are useful, but they answer very different questions.
Head of Frontend Engineering @GoodBarber, teacher & speaker. I build engines that turn no-code into native apps.
Obsessed with apps, UX/UI & AI.
🏔️🎬📚🎲🗺️
speaking of which please review my stock website integralstocks.com i will take any feedback. I am a kid who is just looking for some help actualy getting ppl on my website.
Hey William, happy to take a look. Getting people to visit a new site is tough, so don't get discouraged. I'll check it out and give you some honest feedback on the site itself and what might help attract users.
thanks so much for your feedback. it really helps
Of course! Keep building and experimenting. You're already doing something right by putting your work out there and asking for feedback. That's how you improve. Keep it up!
One feature request: an option to automatically generate profits for people like me who know absolutely nothing about trading. 😂 Jokes aside, really like the idea; inspiring. All the best with the project!
Independence is the missing variable. A review model that shares the author model’s prompt, context, and assumptions can repeat the same blind spot with more confidence. Use a contrast set with hidden defects and measure disagreement by defect class, not just whether the reviewer produces plausible prose.
I like the idea of measuring disagreement by defect class rather than judging how convincing the review sounds. That's a much better way to test whether the reviewer brings something different to the table. A model that writes convincing explanations but misses the same categories of bugs isn't adding much independence.
Exactly. Defect-class disagreement is a much stronger signal than persuasive prose. I would also retain a small set of seeded regressions by class, so the independence claim is tested against known failures rather than inferred only from live review output.
That's a good addition. Seeded regressions give you a repeatable baseline, so you can actually see whether a reviewer improves or starts missing things after a model or prompt change. I'd rather have a reviewer that consistently catches known classes of defects than one that sounds convincing but is impossible to evaluate.
“A sentence about the code isn’t evidence about the code” is the line I’m keeping. Failure mode 2 really hit me too. A new route that looks perfect in the diff but sits outside the auth middleware in a file nobody touched is exactly the kind of bug no diff-only reviewer will ever see, human or AI. Great reminder to ask “which line proves this?” before trusting a confident summary.
That line seems to resonate because it's such a simple test of a reviewer's claims. If it says permissions are enforced, there should be a clear place in the code or a test that proves it. And I agree about the middleware example. Sometimes the important security check isn't in the changed code at all, which is why reviewing only the diff can be misleading.
A reviewer built from the same model and the same context as the author is not a second opinion, it is the same blind spot measured twice. Real independence comes from differing prompts, differing context windows or a different model. Without that, the review stage only adds latency and a false sense of coverage to the merge.
I agree, although I'd add that using a different model alone doesn't guarantee independence either. If both models get the same incomplete requirements, they can still miss the same thing. Different context and a review that starts from the actual security requirements matter just as much. Otherwise we're mostly paying twice for the same assumptions.
That invoice example is going to stay with me. Logged in, ORM clean, everything the reviewer said true — and anyone can still download anyone's invoice. I see the same shape in code-with-AI interviews now. A candidate gets the model to generate something, then asks the same model, does this look right? It says yes, confidently, and they move on. Same context, same blind spot, just with better formatting. The candidates who stand out go back to the requirement instead. Who's allowed to do this? What should never happen? Uncomfortable questions, but that's where the real check lives.
That's an interesting point about AI-assisted interviews. The candidates who question the generated solution instead of just asking the model to validate itself are showing a much more valuable skill. Knowing what questions to ask, especially around what must never happen, is still what separates using AI effectively from blindly trusting it.
the invoice example is the one that sticks, because every sentence the reviewer says is true and the endpoint is still open to any logged in user. imo the fix is moving the check upstream of the model: a test written from the requirement, like alice trying to pull bobs invoice, fails no matter how confident the summary sounds. failure mode 2 is the sneaky one for me, since auth mounted per path in
app.jsnever shows up in the diff at all. do you give your reviewer the route table and middleware order as context, or keep it diff only?I wouldn't keep it diff-only, especially for anything security-sensitive. I'd give it the route table, relevant middleware configuration and order, plus the affected authorization logic. No need to dump the entire repo into context either. Targeted context gathering is the balance I'd aim for. The reviewer should have enough to trace the real request path, not just inspect the changed lines.
AI review was nearly the only check once, on an event app I build. An agent reviewed the new helper role's grant table and tests and reported no permission bugs. I asked for a pass on staging as a seeded helper, clicking through everything. That found a ticket-pricing page showing sales totals to helpers and crew, plus an unrelated double booking. The grants matched the written spec the whole time, and the problems only showed up once a helper opened the screens.
That's a great example of something that's easy to miss when reviewing code in isolation. The grants can be perfectly correct against the spec and you can still have problems in how the app actually behaves. The seeded helper walkthrough is exactly the kind of independent check I had in mind. It tests the experience and the boundaries, not just whether the implementation matches what was asked for.
The invoice endpoint is a perfect example because every statement the reviewer made was true. Login was enforced and the ORM was safe, and any logged in user could still read anyone's invoice. A reviewer that starts from the same ticket inherits the same shortcut.
I agree with your point that the useful check starts at the requirement: who is allowed to read this? That question has to be written down somewhere before review can test against it.
Jackson's seeded helper walkthrough on staging is the kind of check that is independent by construction, since it exercises the running app rather than reading the diff. Alex's idea of measuring disagreement by defect class is a good way to test whether a second model adds anything at all.
On builds at Ascendra Ventures we write the access rules into the acceptance criteria for exactly this reason, so a reviewer, human or not, has something outside the diff to check. Where would you draw the line on letting an AI approval count toward a required review?
I'd draw the line at whether the approval provides genuinely independent assurance. For documentation or formatting changes, I'm comfortable with AI approval counting as a lightweight check, provided the scope is restricted. For anything involving auth, permissions, payments, secrets or personal data, I'd always require a human approval. The important thing is that we don't let a green AI status quietly become a substitute for someone taking responsibility for the merge.
The invoice endpoint example is the one that got me — auth enforced, ORM clean, and still the wrong person's invoice. That's exactly the blind spot a second model shares, because it read the same ticket. The seeded-helper walkthrough in the comments feels like the only check that actually breaks the loop. Do you do that kind of walkthrough on every role change, or just the ones that touch money?
I'd say every role change deserves at least some level of walkthrough, not just the ones touching money. Permissions tend to have side effects that aren't obvious from the code. The depth of testing can vary with the risk, but I'd always test what the new role can actually see and do, including what it shouldn't be able to access.
The invoice example is the one that stuck with me too — every statement the reviewer made was true, and the actual authorisation question was still unanswered. That distinction between "logged in" and "allowed to read this invoice" is easy to miss when the review starts from the diff.
I also appreciated failure mode 2. A new route can look completely clean on its own while the auth middleware that should cover it lives in a file the PR never touches. Starting from the request path, rather than the changed lines, feels like the practical takeaway here.
Exactly. Following the actual request path changes the whole review. The diff tells you what changed, but it doesn't necessarily tell you which security controls apply to it. That's why I think tracing the flow should be part of reviewing any security-sensitive change.
Your rule that AI review should never count as an approval stood out to me. GitHub’s new Copilot approval option goes the other way: it can now count toward required reviews. It’s off by default and admins choose the file paths, but if AI wrote the code and AI also approves it, aren’t we essentially counting the same opinion twice? Would you consider enabling it for low-risk changes like documentation, or keep it disabled entirely?
I think documentation is a reasonable place to experiment with it, as long as the scope is tightly controlled. But I'd still distinguish an AI approval from a human approval in the process. My concern isn't that AI review has no value, it's that we start counting it as independent assurance when it might not be. The label matters less than being honest about what was actually checked.
Yes. The same weak requirements give the same blind spot. Independence comes from the context, not the model name.
Exactly. You can swap models all day, but if neither is given the right questions to ask, the result won't change much. The reviewer needs something independent to compare the code against, not just another copy of the same assumptions.
The invoice example is the one that sticks with me — everything the reviewer said was true, and the access problem was still there. “Logged in” isn’t the same as “allowed to see this record,” and that distinction matters a lot when the record is a client file, a lease, or a payment.
I like your rule that AI review is advisory, never an approval. For anything touching auth or money, I want a person checking it against the actual requirement, and a test written from that requirement — including the test where the wrong person gets refused. That’s the check that actually catches this one.
Completely agree. The negative test is particularly important because happy-path tests can make a broken authorization check look perfectly healthy. Alice accessing her own invoice proves very little about whether Bob's invoice is protected. I'd want both the human review and that explicit refusal test before merging anything involving sensitive records.
For highly vulnerable features as you mentioned, such as payments, authorization, authentication, etc., it's essential to keep a human in the loop rather than blindly trusting what the model says. For some lower-risk changes, we can allow it though.
Also, we can't just ask the model to go through the entire repo's code and check whether the PR is aligned with the requirements, vulnerabilities, and so on. Otherwise, the token usage and cost would be very high. There has to be a balance, which obviously a human has to make. Targeted context gathering seems like a more practical approach.
Switching different models for writing and reviewing the code will help identify the first model's mistakes and blind spots, but the review still can't be blindly trusted.
One nightmare: if a human is asked to review the entire PR work done by an AI model for a project which is 90–95% built using AI. 💀
If an agent is writing the code and another agent is reviewing it, the chances are both agents might share the same blind spots, effectively automating both the introduction of bugs and their approval.
That 90-95% AI-built project scenario is probably where this gets really interesting. At that point, the human reviewer isn't just checking a PR, they're trying to understand a system they may not have designed or fully explored. I agree that targeted context is the practical approach. Start with the changed behavior, trace its dependencies and security boundaries, then bring in the relevant tests and requirements. Different models can help, but someone still needs to own the final judgment.
Different model is not enough. A second review helps when it starts from the threat model instead of the diff.
Exactly. Starting from the diff means you're mostly checking the implementation. Starting from the threat model means you're questioning whether the implementation is safe in the first place. That's a much more useful kind of independence, regardless of which model is doing the review.
Right. A review that starts from the threat model asks what can go wrong before it asks what the code does.
That's the distinction. One checks whether the code matches the intended implementation, the other checks whether that implementation is actually safe. Both are useful, but they answer very different questions.
Exactlyyy
Haha, glad we're on the same page! 😄
Do you give the AI reviewer the ticket at all?