Where We Stand — May 2026
An honest, forward-looking update for leadership. Click “View details” on any section to see the full breakdown.
We Were Hit — And We Are Better for It
View details ›
Two incidents in May 2026 exposed gaps we might never have
found otherwise. We moved quickly — three critical fixes are already
live, two more ship by June 8. Our platform is now more secure than it has ever been, and we have a clear plan to keep it that way.
3 Fixes Live
2 Remaining — Jun 8
Our Growth Demands a Bigger Engine — We Are Designing It
View details ›
2.5 million users and growing fast. The platform needs to
handle more — we are reviewing every critical area and redesigning for
the next stage. A quick performance optimisation ships next sprint so users feel the improvement immediately. We are planning load testing to validate the app under 500 to 1,000 concurrent users, and every planned change is designed to increase how many people the platform can support at once.
Quick Optimisation — In Progress
Load Testing Planned
Full Architecture Underway
Cloud Security Is Being Wired Into the Foundation, Not Bolted On
View details ›
The incidents revealed our cloud security posture needed a
serious upgrade. We identified every gap and are closing them
systematically. Many are already done — every remaining item has an owner and a deadline.
Active Remediation in Progress
Application Security — Reviewing and Strengthening Every Layer
View details ›
Beyond the incidents, we are reviewing our application-level
security practices — how data is protected, how sessions are managed,
how files are handled, and how access is controlled. Some areas are already solid; others are being reinforced right now.
Review In Progress
How We Propose to Build and Deliver Going Forward
View details ›
We are recommending a unified engineering framework for how
both Canada and India teams plan, build, and ship — with clear quality
standards, documented decisions, and automation in every sprint. We believe this is the right operating model for DogPack’s scale, and we are proposing it for adoption.
Framework Proposed — Pending Alignment
Critical Infrastructure Incidents & Action Plan
Two critical production incidents occurred in May
2026. Remediation is actively underway. Immediate management attention
required on all open items.
🚨
Incident 1 — Security Breach — Media Content Downloaded by Attackers
Attackers used an automated script to exploit raw CloudFront URLs hardcoded in the application, downloading a large volume of media content while bypassing CloudFlare security controls.
Root Cause
- Internal CloudFront distribution URLs were directly embedded in the codebase instead of routing through CDN or custom domain
- These URLs bypassed CloudFlare WAF, DDoS protection, and rate limiting entirely
- No expiry or signing mechanism was in place — captured URLs remained permanently valid
5-Point Remediation Plan
Code Audit & Route via CloudFlare — At Website
Fix 01
Done
- Full codebase scanned — all raw CloudFront URLs identified and replaced
- All media requests now routed via CloudFlare / CDN custom domain
- Zero internal media URLs exposed in production
Pre-Signed & Temporary URL
Fix 02
In Progress
- Code scan and changes for temporary URL — In Progress
- CloudFront URL is not being exposed to users or attackers
- DoneAll media uploads served via pre-signed URL — every file has a time-limited signed link
WAF & Rate Limiting
Fix 03
Done
- CloudFlare rate limiting enforced at edge
- CloudFront WAF deployed as secondary defense layer
User Agent & Header Validation
Fix 04
In Progress
- Analysis underway via CloudFlare to allow only valid requests with recognised user agents and headers
- Requests from unknown or invalid clients to be rejected before reaching origin
- ETA June 8, 2026
Impact Analysis In Progress
- Renaming CloudFront distribution so any previously captured URLs return 404
- Impact analysis underway — SEO and WordPress dependency being assessed before proceeding
- ETA June 8, 2026
🔴
Incident 2 — Database Outage: MySQL Read Replica Unresponsive
The read replica became completely unresponsive, causing data load failures for users. Manual restart was required. No confirmed single root cause — multiple workloads were running in parallel at the time.
Contributing Factors (All Simultaneous)
📈
Application Traffic — Normal user-facing read load on the replica
🤖
Google Bot Crawling — Aggressive indexing amplifying read queries significantly
🔄
Shopify Sync — Background sync jobs competing for CPU on the same instance
⚡
Result: All three workloads running simultaneously — the read replica became unresponsive, requiring manual restart
Action Plan & Current Status
| Action Item |
Status |
Notes |
| Database Instance Analysis |
Done |
Analysis complete — instance sizing and capacity reviewed
Recommendation: Upgrade to a higher instance type to handle combined peak workloads
|
| Database Instance Upgrade |
Pending |
Upgrade read replica to handle combined peak workloads
Prevents recurrence under concurrent load scenarios
|
| Feed Architecture Review |
Done |
Complex feed logic must be offloaded from MySQL
Move ranking and discovery to Elastic / OpenSearch
|
| Offload Feed to Search DB (OpenSearch) |
Solution Scoped — Ready to Plan |
We have a clear approach in mind — moving feed ranking to a dedicated search engine to take load off the main database
Plan is ready; execution timeline being confirmed
|
| Last 30-Day Feed Data in Cache |
Solution Scoped — Ready to Plan |
We have an approach identified — keeping recent feed data
in a fast cache so the database is not hit for the same content
repeatedly
Plan is ready; execution timeline being confirmed
|
✓ Observability — App-Side Logs Enabled (May 21)
- Datadog enabled on May 21 — app-side logs are now active and capturing application events
- This was a key gap during the database incident — root cause diagnosis will be significantly faster going forward
- Distributed tracing across services is next — requests will soon be fully correlated end-to-end
- E2E observability scoping in progress Immediate Priority
Scaling H2 2026 — Architecture Review
With rapid user growth, a full architecture
review is underway across all critical features — not limited to the
database. The goal is to ensure every system layer can support
concurrent load, eliminate single points of failure, and sustain growth
beyond current scale.
Architecture Review of Critical Features — In Progress
A comprehensive review is being conducted across feed
loading, search, database, real-time calculations, and observability to
identify bottlenecks and improvement scope at scale.
Improvement Scope
1. Feed Load Optimisation
- Current: Feed calculated on-demand per request — slow under concurrent load
- Quick Solution Proposed: Fetch top 100 feed IDs upfront in a single lightweight call — show first 10 immediately
- On scroll, fetch details for the next 10 IDs from the already-fetched list — no extra DB call until page 11
- From page 11 onward, fetch the next batch of 100 IDs and repeat the cycle — keeps DB queries predictable and infrequent
- Long-term: Analysis in progress to move feed ranking to OpenSearch + Redis cache to further reduce MySQL load
2. Video Streaming Quality
- Current: Single resolution (720p only), heavy compression, 30 sec–2 min latency before video is viewable
- Solution: Adaptive video streaming with multi-resolution output (480p, 720p, 1080p) and adaptive bitrate player
Solution Scoped — Ready to Plan
- Current: User search and dog search running expensive wildcard queries directly on MySQL
- Improvement: Move user search and dog search to a dedicated search database (OpenSearch)
- Offloads heavy search load from MySQL — faster results and reduced DB pressure
Analysis In Progress — Ongoing
- Translations logic reviewed — scalable logic finalised, POC pending
- Database instance upgrade — analysis in progress, discussion needed on instance sizing
- Reviewing overall DB architecture to support higher concurrent connections
5. Follow / Unfollow Performance
- Performance issue identified with follow and unfollow operations
- Both database and code-level analysis pending to identify root cause
- Evaluating whether a graph database approach is suitable for follower relationships at scale
6. Load Testing & Concurrency Validation
- Validate application performance across critical flows under 500, 800, and 1,000 concurrent users
- Run tests against current fixes already deployed to confirm stability, then re-test as each optimisation lands
- Feed, search, and DB improvements will significantly offload MySQL — load tests will confirm the compounded improvement in response time and concurrency capacity
- Identify any remaining bottlenecks before they become production incidents
Delivery Timeline
Dates marked Committed are agreed and in execution. All other dates are indicative and confirmed sprint by sprint.
Delivered / Committed
In Progress
Solutioning
Development
Testing
Each column ≈ 2 weeks · May 25 → Sep 28
May 25
Jun 8
Jun 22
Jul 6
Jul 20
Aug 3
Aug 17
Aug 31
Sep 14
Sep 28
✅ Delivered
Android 16 KB UpgradePerformance improvement
API Optimisation Phase 1Technology upgrade
⚡ Quick Optimisation Items
DB — Dedicated Feed Read ReplicaAurora · TBC
Feed Load — Quick OptimisationJun 4
Park Search — Quick OptimisationDone
User & Dog Search → OpenSearchJun 22
Park Search → OpenSearchJun 22
🔧 Active Deliveries — Indicative
API Optimisation Phase 2Technology upgrade
Unity to NativeApp stability & performance
⚡ Scaling — Indicative
Feed Load — Long TermOpenSearch + Redis
Video Streaming QualityPhase 1 — HD + adaptive
Database UpgradeIf Needed
Follow / UnfollowDiscussion Pending
⚠ Green bars in Committed section have agreed dates. All other bars are indicative — confirmed sprint by sprint.
How These Changes Work Together
Combined DB offload impact: Feed load optimisation
(OpenSearch + Redis), search migration (OpenSearch), and caching
improvements will collectively and significantly reduce read load on
MySQL — making the database faster and more resilient for all remaining
operations. Each improvement compounds the benefit of the others.
Target State
- Working on all the above items will drastically improve concurrent user capacity and overall app performance
- Feed, search, and follow operations become fast and predictable regardless of load
- Load testing will validate and quantify the improvement at each phase
Infrastructure Resilience
- No single point of failure — load spread across purpose-built systems (MySQL, OpenSearch, Redis)
- MySQL relieved of read-heavy workloads — reserved for writes and critical operations only
- Architecture built to sustain growth well beyond current user base
AWS and Cloud Security
Comprehensive security controls across IAM, network hardening, VM patching, threat detection, and cost optimisation across all AWS workloads.
Security Optimisation
Done
- VPN implemented for Stage and Production environments — Done
- All open SSH ports removed — access to instances restricted through VPN only — Done
- Production database access restricted through VPN only — Done
CloudFlare Traffic Restrictions
- HTTP/HTTPS traffic restricted to Cloudflare IP ranges only — Done
- ALB access restricted to Cloudflare IP ranges only — Done
- All Cloudflare domains are proxied — except Chat domain — Done
Security Group Review & Cleanup
- Full Security Group review and cleanup completed — Done
- Application ports restricted to required source IPs / Security Groups only — Done
- All unnecessary public access removed — Done
AWS CloudTrail & Audit Logging
- AWS CloudTrail — management events and full API audit logging enabled — Done
- All API calls, console sign-ins, and resource changes are logged and retained — Done
AWS GuardDuty — Production
- AWS GuardDuty enabled for Production environment — continuous ML-based threat detection active — Done
- Alerts configured for suspicious activity, compromised credentials, and unusual API calls — Done
AWS Config — Resource Compliance
- AWS Config enabled for resource configuration recording — Done
- Compliance management active — tracks configuration changes across all resources — Done
Done
- VPC Flow Logs enabled for network monitoring and auditing — Done
- All inbound and outbound traffic at the VPC level is captured and retained for review — Done
Done
- EBS volume encryption enabled for all new volumes — Done
- Encryption at rest enforced by default — no unencrypted volumes provisioned going forward — Done
Done
- EC2 Instance Metadata Service (IMDSv2) enforced on all instances — Done
- IMDSv1 (unauthenticated metadata access) disabled — prevents SSRF-based credential theft — Done
Done
- MFA enforced for all users on AWS Console — Done
- Old IAM access keys cleaned up — Done
- Regular IAM access key rotation schedule established — Done
WAF, CloudFront and DDoS Protection
- Lambda-based automated monitoring — detects and blocks suspicious IPs — Done
- Origin Access Control (OAC) configured — Done
- AWS WAF attached to CloudFront with rate limiting and managed rule sets — Done
- WAF and security rules reviewed and cleaned up — In Progress
Done
- Google Crawler Bot rule configured — Done
- cURL block at domain level — Done
- cURL block at application endpoint — Done
In Progress
- SSH root login disabled on all instances — Done
- OS patching managed via AWS Patch Manager — Pending
Pending
- Existing ECS infrastructure upgrade — Pending
Pending
- AWS access key and secret rotation — Pending
Cost Optimisation
In Progress
- All unused EC2 instances stopped — Done
- Decommissioning requirements being evaluated to eliminate ongoing costs — In Progress
Unused Read Replica — Stage
- Unused Read Replica removed from Stage environment — Done
- Reduces unnecessary RDS compute and storage costs — Done
Temporary Blog RDS — Production
- Temporary blog RDS (wordpress-blog-tmp-db) instance stopped — In Progress
In Progress
- Continuous review of unused AWS resources across all environments — Ongoing
- Goal: eliminate idle resources, right-size active ones, and reduce operational costs month on month
Application Security Best Practices
Industry-standard guidelines for all AWS-hosted
applications across 10 core domains, based on OWASP Top 10, AWS Security
Reference Architecture, NIST SP 800-53, and CIS Benchmarks.
OSS Governance and Approval
- All packages under review — Legal and Compliance approval process being established
- Tooling: FOSSA for license compliance scanning across all dependencies
- Restrict AGPL and commercial dual-license components from production use
Already Implemented
- Cryptographically random tokens — 128-bit entropy minimum
- Server-side invalidation on logout — clearing cookie alone is insufficient
Data Protection and Encryption
- AES-256 encryption at rest for all sensitive data
- TLS 1.2 minimum for all data in transit
- Credentials never hardcoded — rotate secrets on a defined schedule
- File scanning — Pending
- MIME type and extension allowlisting — Already Implemented
Implemented — Detailed Review In Progress
- Validation implemented at entry points — currently being validated in detail to ensure full coverage
- Output encoding at render time — both layers required
- CSP, HSTS, X-Frame-Options, CORS on every HTTP response
Review In Progress
- Access control checks are present in the codebase
- Reviewing whether controls are correctly enforced at the backend layer or only at frontend — backend enforcement is mandatory
- Ensuring no data can be accessed by bypassing the UI directly via API
Delivery & Team Excellence
A proposed framework for how Canada and India
teams plan, build, and ship together — one process, one standard, one
product. Four pillars: Visibility, Quality, Automation, and Scalability.
- Canada — Product website, client features, customer coordination, business requirements
- India — App and backend, infrastructure, DevOps, platform engineering, performance
- Daily Scrum (15–30 min) across both teams — blockers, risks, integration topics
- Weekly Architecture and Design Sync for deep technical alignment
Jira & Confluence — Single Source of Truth
- No Jira ticket = No work — every change, no matter how small
- Full hierarchy: Epics → Stories → Sub-tasks → Bugs and Risks
- Both teams, one board, full transparency
- Severity-classified bugs triaged in every daily scrum
- All documentation, decisions, and runbooks maintained in Confluence
- Clear Scope — every dev, QA, BA must articulate it in one sentence
- Technical Approach — API contracts and integration points made explicit
- Impacted Areas — all affected modules listed upfront (most critical element)
- Test Cases written before development starts (shift-left)
- BA and QA sign-off required before any story is closed
- Mandate: Build it, automate it, in the same sprint — no deferrals
- Flow: Groom → Dev → QA → Automation Written → Evidence Captured → Story Closed
- Testing evidence required on Dev and Staging environments
Architecture Design Records (ADRs)
- Every significant architectural decision must be documented as an ADR in Confluence
- Captures context, options considered, decision made, and trade-offs — permanent reference for the team
- Prevents repeated debate on settled decisions and ensures new team members have full context
- ADRs linked directly to the relevant Jira epic or story
Four Pillars:
Visibility — every work item in Jira
· Quality — shift-left testing, sign-off required
· Automation — in-sprint, no deferrals
· Scalability — scale considered in every design decision from day one
NineHertz Support Coverage & Engagement
Introducing Rohit Rao (CTO, NineHertz) and Arpit (Cloud Security Consultant) to monitor progress, align priorities, and accelerate delivery across security and infrastructure.
🎤
NineHertz CTO
Rohit Rao
CTO Introduction
Rohit Rao to be introduced to align on support expectations, ongoing priorities, and delivery direction across all active workstreams.
✓ Aligns delivery direction
✓ Establishes ongoing touchpoint
✓ Single escalation point for CRO & CEO
🛡
Cloud Security Consultant
Arpit
Security Coverage
Arpit
included in all security discussions to ensure coverage is aligned with
planned allocation and remediation is actively monitored.
✓ Planned allocation honoured
✓ Security ownership clarified
✓ Incident remediation tracked end-to-end
👥
Day-to-Day
Working Coordination
NineHertz
team works closely with DogPack on day-to-day coordination, ensuring
visibility into open items, blockers, and progress across all
workstreams.
✓ Direct line for execution
✓ Faster issue resolution
✓ Progress visible to CRO & CEO at all times
Action Plan — Live
Specific actions agreed in the May 26 session. Owners named. Dates committed. This is the plan we are executing right now.
✅ Completed
Feed API — Query Optimisation
Done
VirendraPunit
Feed API queries optimised — reduces database load and improves feed load speed for all users.
Impact: Faster feeds, lower DB pressure. Phase 2 follows after load test data.
User Search — Query Optimisation
Done
VirendraPunit
User search queries optimised at the MySQL level — expensive wildcard scans reduced ahead of the OpenSearch migration.
Impact: Less DB load. Faster user search results today.
Park Search — Query Optimisation
Done
Punit
Park search queries optimised and monitored — quick wins applied ahead of the full OpenSearch migration.
Impact: Every query optimised reduces DB load today.
Admin panel User List is live — team can view and manage all registered users.
Query Optimisation: User listing queries optimised — paginated efficiently to handle large user volumes without DB strain.
Admin — Business List with Search
Done
Admin panel Business List with Search is live — team can find and manage business accounts quickly.
Query Optimisation: Business search queries optimised — indexed lookups ensure fast results even as the business account list grows.
IP Address Block via CloudFlare
Done
Known attacker IPs blocked at the CloudFlare edge using WAF rules — no origin traffic reaches the application.
Impact: Bad actors stopped at the edge before reaching the app.
Security & Cost Optimization
Done
Security Optimization
✓Secure private network (VPN) enforced for all staging and production access
✓All public server entry points closed — only authorised team members can connect
✓Web traffic filtered through Cloudflare — direct attacks on servers blocked
✓All app domains routed through Cloudflare for DDoS protection and WAF coverage
✓Production database fully isolated — accessible only via secure private network
✓Full audit trail enabled on infrastructure — every action is logged and traceable
✓Infrastructure compliance monitoring active — configuration changes are tracked
✓Automated threat detection enabled on production — suspicious activity flagged in real time
✓Access permissions reviewed and tightened — unused or overpermissive rules removed
✓Network traffic monitoring enabled — all activity logged for security review
✓Data at rest encrypted — all new storage is protected by default
✓Server access hardened — metadata exposure vulnerability mitigated
Cost Optimization
✓Stopped unused EC2 instances
✓Removed unused Stage Read Replica
✓Removed temporary blog RDS (wordpress-blog-tmp-db) from Production
⏳Ongoing unused resource cleanup — In Progress
Impact: Attack surface reduced across network, compute, DB & storage. CloudTrail, Config & GuardDuty now active. Unnecessary infrastructure costs — cleanup in progress.
Load Testing — Round 1
Done
Completed
First performance test completed. Simulating realistic user journeys at 2 new users per second (ramp-up) across key flows:
User & Profile
Register
Self Profile
Dogs
Add Dog
My Dogs
Dog Profile
Dog Media
Parks
Search
Park Detail
Gallery
Amenity Ratings
Weather
Feeds
Feed Algo v2
Dog Feeds
Park Feeds
| Metric |
Before |
After |
| Optimised API Testing (excl. Reels) |
~300 users |
2,000+ users |
| Database Timeout / Load |
Yes (frequent) |
None |
| Stable Capacity — Under Basic Load |
< 300 |
1,000–1,500 |
| Reels under load |
Not tested |
In progress |
| Round 2 target |
3,000+ users (incl. Reels) |
What this means: With average usage patterns the platform comfortably supports 1,000–1,500 concurrent users today. Heavy parallel Reels usage may cause slowness at higher loads — addressed in the 30 June UAT release. Round 2 targets 3,000+ users including Reels.
🚀 Releasing — UAT Deploy · 30 June
The following items are will be deployed to UAT on 30 June. Not yet in production.
User Onboarding
UAT · 30 June
Complete user onboarding suite — all flows built and deploying to UAT on 30 June.
Login
Social Login / Register
Register
Add Dog
Performance: Several areas identified during load testing are being improved as part of this release.
Feed Algorithm
UAT · 30 June
Feed ranking and discovery algorithm — content surfaced based on relevance and recency.
Query Optimisation: Feed queries restructured to reduce DB round-trips and improve load time under concurrent users.
Park Detail
UAT · 30 June
Park detail page — full park info, photos, and check-in data.
Query Optimisation: Park detail queries consolidated — fetches all required data in fewer, more efficient DB calls.
Reels Page — Migration & Optimisation
UAT · 30 June
Reels page migration deploying to UAT on 30 June. Excluded from Round 1 load testing. Confirmed performance numbers will be shared once UAT is complete.
Impact: Reels is the last major feature not yet load-tested. Its inclusion in Round 2 gives a complete picture of platform behaviour under full concurrent load.
🔧 In Progress
Dedicated Read Replica — Feed API Only / Aurora
In Progress
✓
Step 1 — Version Upgrade
Completed on UAT — Verification in progress
⏳
Step 2 — Migration to Aurora
Pending — in planning
Impact: Dedicated replica for Feed API isolates feed load — prevents it from affecting the rest of the platform.
Load Testing — Round 2
In Progress
TBD — post Reels
Round 2 targets 3,000+ concurrent users including the Reels API once migration is complete. Same realistic lifecycle: Registration → Dog Profile → Dog Search → Park Search → Like / Watch / Post → Reels. Full report shared with the entire team.
Impact: Validates headroom well beyond Round 1. Confirms whether Reels at scale introduces new DB pressure.
Admin Dashboard — Real-Time Counters Fix
In Progress
Admin screen currently computes user/post counts in real-time on every load, putting heavy query load on the DB. Replacing with pre-computed counts stored in a table — updated asynchronously.
Impact: Eliminates a recurring heavy query pattern from an internal tool. Reduces background DB pressure invisible in production monitoring.
Use Case Load Review
Ongoing
A dedicated review to identify any user-facing flows introducing unexpected database load — flows not yet covered in our load test scenarios.
Impact: Ensures no blind spots remain. Product-informed load testing is more accurate than infrastructure-only analysis.
🔁 Ongoing — Every Release
Security & Performance Testing — Built Into Every Release Cycle
Security hardening and performance testing are now a standing part of our release process. After every release, we run performance and load tests to understand how the system behaves with increasing load, new features, and new security considerations.
A full report is shared with the entire team after every round — so leadership, product, and engineering all have a shared, data-driven baseline for every major decision.
After every release
Report shared with full team
Security + Performance
🔎 OpenSearch Migration — TBD
Queries for User Search and Park Search have already been optimised at the MySQL level and are being continuously monitored. The full migration to OpenSearch will be planned once the load test results confirm the next priority. Date is TBD.
User Search → OpenSearch
TBD
VirendraTBD
Queries already optimised and being monitored continuously. Full migration from MySQL to OpenSearch will eliminate wildcard query load entirely — date to be confirmed after load test.
Impact: Faster search. Significantly less load on MySQL.
Park Search → OpenSearch
TBD
RanjeetTBD
Queries already optimised and being monitored continuously. Full migration from MySQL to OpenSearch will remove complex location-based queries from the main database — date to be confirmed after load test.
Impact: Removes two of the heaviest recurring query patterns from MySQL.
📈 Long-Term — Ongoing
Feed Data Architecture — Long-Term Solution
Ongoing
DevrajVishalVirendra
Move recent feed data (last 30 days) to Redis or OpenSearch — takes the heaviest read load off MySQL. Fallback to MySQL feed always available.
Impact: Could dramatically reduce DB read load if usage data confirms the hypothesis.
Milestone Summary
Done — Live
Deploying to UAT · 30 June
In Progress
Planned
Done ✓
Feed API
Query Optimisation
Done ✓
User Search
Query Optimisation
Done ✓
Park Search
Query Optimisation
Done ✓
Admin Business
List + Search
Done ✓
IP Block
via CloudFlare
Done ✓
Security & Cost
Optimization
Done ✓
Load Test Round 1
1,000–1,500 users
30 Jun →
Social Login /
Register
In Progress
Dedicated Feed
Read Replica
In Progress
Load Test Round 2
3,000+ users
TBD
User + Park
→ OpenSearch
Ongoing
Feed Architecture
Long-Term
Load Test Round 1 ✓ Complete (1,000–1,500 stable · 2,000+ peak). Next: UAT Deploy · 30 June → Reels Migration → Load Test Round 2 — targeting 3,000+ users.
