AI SaaS security / Assessment method
Make the scope and evidence inspectable.
This method supports our AI SaaS security audit. The engagement scope identifies the applicable tests, access, evidence, and deliverables. This page is a description of the process, not an assessment of any product.
1. Agree the boundaries before testing
Record the product version, environment, target domains and APIs, model configuration, permitted accounts, and third-party systems. Obtain written authorization and agree test windows, stop conditions, data handling, and escalation contacts.
Use synthetic or approved data and bounded tests. Third-party infrastructure needs its own authorization where applicable. Code review, configuration review, application testing, and AI evaluation are included only as agreed; inaccessible components remain an explicit limitation.
2. Separate observations from statements
- Tested: a recorded input, environment, configuration, and observed result support the finding.
- Reviewed: code, settings, or documents support a narrower conclusion within the available access.
- Declared: a provider or product owner states a practice that Rubrex could not independently inspect.
- Not assessed: the check was excluded, unavailable, or inconclusive. It must not be reported as a pass.
For a training-use claim, identify which data, provider, account configuration, terms, and time period the evidence covers. A settings review cannot establish every internal action of an external provider.
3. Prioritize, fix, and retest
Prioritize findings using demonstrated impact, affected users or data, prerequisites, and the assessed environment. Keep the original evidence and record the fix, retest conditions, result, and remaining uncertainty. An accepted risk stays visible as an accepted risk.
Evaluate quality separately using agreed reference cases and scoring criteria. Report sample coverage and known limits instead of converting a small test set into a universal accuracy claim.
Illustrative report outline / Not a completed audit
What an assessment report contains
This outline shows the intended structure. It contains no client results, assessed product, or passing badge.
- Executive summary: the decision being supported, significant findings, and next actions.
- Scope: product version, environment, dates, authorized targets, criteria, and exclusions.
- Data-flow map: customer input → application → retrieval or tools → model provider → output, with logs, analytics, storage, retention, and deletion paths identified.
- Finding records: identifier, affected component, evidence type, expected versus observed behavior, impact, severity rationale, and remediation.
- Evaluation results: cases, grading method, quality findings, and coverage limits.
- Retest record: updated version, repeated checks, outcome, and unresolved issues.
- Public summary, if agreed: supported statements with sensitive implementation details removed.
Reproduction details, secrets, and information that could expose customers belong in the restricted technical report, not a public badge page.
Requirements for any Rubrex assessment badge
The proposed badge wording is “Security & data practices assessed by Rubrex.” A badge is not issued merely because an assessment was purchased. Issuance requires a completed review and a published verification record.
- The product owner’s authority and the assessed product/domain must be verified.
- Required checks and pass criteria must be agreed before results are known. Required checks cannot be silently removed to obtain a pass.
- Unresolved critical or high security findings prevent an active badge. Other findings and exclusions must be disclosed at a safe level.
- Each public data-practice claim must have supporting evidence and a clearly stated boundary. Unsupported claims are omitted.
- Remediation must be retested where needed before eligibility is confirmed. AI or scanner output alone is insufficient evidence.
- Any Rubrex engineering work on the assessed product must be disclosed. The assessment cannot imply independence from work we performed.
A badge must link to a current record
The verification record must identify the product and domain, assessment ID, scope, assessed version, dates, evidence summary, limitations, expiry, and status. Anyone viewing a badge must be able to check that record on rubrex.ai.
Validity is limited to the stated scope and expiry. Changes to providers, data flows, authorization, model tools, or other assessed controls require review of the affected claims. An expired, suspended, or revoked assessment cannot be displayed as active. Renewal requires fresh evidence, not a changed date alone.
The badge does not establish SOC 2, ISO certification, legal compliance, zero vulnerabilities, or that a product never processes customer data. An assessment is bounded evidence about the tested system, not a promise about every future request.
If you need to check a claimed Rubrex assessment, send the product URL and assessment ID to evals@rubrex.ai. Do not send private reports or credentials through the public contact form.
References used to define the scope
These sources inform the assessment design. They do not certify Rubrex or an assessed product. The report must list the versions and applicable requirements actually checked.
- OWASP Application Security Verification Standard 5.0.0: a reference for application security requirements.
- OWASP Top 10 for LLM and GenAI applications: a reference for model-application risk coverage.
- OWASP Top 10 for Agentic Applications: a reference for agent behavior, tools, and privilege boundaries.