Drop stale old RBAC FK indexes from model tables#16392
Conversation
📝 WalkthroughWalkthroughModified ChangesImplicit role index removal
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (2)
awx/main/tests/functional/migrations/test_0206_remove_old_rbac_indexes.py (2)
7-8: Consider removing@pytest.mark.django_dbif no DB access is needed.The tests only inspect the migration module's structure and don't interact with the database. The decorator adds overhead by setting up database transactions for each test.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@awx/main/tests/functional/migrations/test_0206_remove_old_rbac_indexes.py` around lines 7 - 8, The class-level `@pytest.mark.django_db` on TestRemoveOldRbacIndexes is unnecessary if none of its tests access the database; remove the decorator from the TestRemoveOldRbacIndexes class (or move the marker to only those individual test methods that require DB access) so tests that merely inspect the migration module's structure don't incur DB setup overhead.
66-71: Dead code:partsandtablevariables are unused.Lines 67-68 compute
partsandtablebut the actual table extraction logic on lines 69-71 ignores them entirely, relying instead onidx.startswith(expected).♻️ Proposed simplification
def test_covers_all_model_tables(self): expected_tables = { 'main_credential', 'main_instancegroup', 'main_inventory', 'main_jobtemplate', 'main_organization', 'main_project', 'main_team', 'main_workflowjobtemplate', 'main_activitystream_role', } actual_tables = set() for idx in self.migration_module.OLD_RBAC_FK_INDEXES: - parts = idx.rsplit('_role_id', 1) - table = parts[0].rsplit('_', 0)[0] for expected in expected_tables: if idx.startswith(expected): actual_tables.add(expected) assert actual_tables == expected_tables🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@awx/main/tests/functional/migrations/test_0206_remove_old_rbac_indexes.py` around lines 66 - 71, The variables parts and table computed from each idx in migration_module.OLD_RBAC_FK_INDEXES are unused; remove the dead computations (the lines that set parts = idx.rsplit('_role_id', 1) and table = parts[0].rsplit('_', 0)[0]) and keep the existing startswith-based check against expected_tables that populates actual_tables, ensuring no other code depends on parts/table.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@awx/main/tests/functional/migrations/test_0206_remove_old_rbac_indexes.py`:
- Around line 7-8: The class-level `@pytest.mark.django_db` on
TestRemoveOldRbacIndexes is unnecessary if none of its tests access the
database; remove the decorator from the TestRemoveOldRbacIndexes class (or move
the marker to only those individual test methods that require DB access) so
tests that merely inspect the migration module's structure don't incur DB setup
overhead.
- Around line 66-71: The variables parts and table computed from each idx in
migration_module.OLD_RBAC_FK_INDEXES are unused; remove the dead computations
(the lines that set parts = idx.rsplit('_role_id', 1) and table =
parts[0].rsplit('_', 0)[0]) and keep the existing startswith-based check against
expected_tables that populates actual_tables, ensuring no other code depends on
parts/table.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: c81d98a5-c4d4-454d-b3a2-f4f501cd21e5
📒 Files selected for processing (2)
awx/main/migrations/0206_remove_old_rbac_indexes.pyawx/main/tests/functional/migrations/test_0206_remove_old_rbac_indexes.py
AlanCoding
left a comment
There was a problem hiding this comment.
I like the spirit of this, but I have been bitten before by a mismatch between the Django field definition and what's in the database. Dropping in raw SQL is likely to cause that mismatch and the migration logic will often error long in the future when the next change is made.
AFAIK, the more correct way to do this is apply db_index=False to those fields when set on the fields. In this case, it is a custom field that inherits from ForeignKey, and then makemigrations should pick that up and drop these indexes on their own. This approach is likely to get us in a major bind in the future.
|
Cheers @AlanCoding , spot on. Raw SQL would definitely leave Django field state out of whack with the database, and that'd come back to bite us on the next migration touching these fields. I've reworked the PR to go the route you suggested:
Tests have been rewritten to validate the |
b94a36d to
a511e32
Compare
After the transition from legacy AWX RBAC to DAB RBAC (migration 0192+), the ImplicitRoleField FK columns on model tables retain 38 indexes that are never scanned. These indexes accumulate through upgrades and add unnecessary write and VACUUM overhead. Set db_index=False on ImplicitRoleField.__init__ and add migration 0206 with 38 AlterField operations across Organization, Team, Credential, Project, Inventory, JobTemplate, WorkflowJobTemplate, and InstanceGroup. This keeps Django's migration state in sync with the actual database schema and avoids the mismatch that raw RunSQL would cause. Signed-off-by: Alexey Masolov <amasolov@redhat.com> Signed-off-by: Alexey Masolov <alexey.masolov@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: Alexey Masolov <alexey.masolov@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com>
a511e32 to
e895059
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (1)
awx/main/tests/functional/migrations/test_0206_remove_implicit_role_field_indexes.py (1)
21-27: ⚡ Quick winAssert operation count directly to avoid vacuous passes.
Current checks can still pass if
Migration.operationsis empty. Add a direct count assertion tied toIMPLICIT_ROLE_FIELDS.Proposed test hardening
def test_migration_targets_correct_field_count(self): assert len(self.migration_module.IMPLICIT_ROLE_FIELDS) == 38 + assert len(self.migration_module.Migration.operations) == len(self.migration_module.IMPLICIT_ROLE_FIELDS)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@awx/main/tests/functional/migrations/test_0206_remove_implicit_role_field_indexes.py` around lines 21 - 27, The test can vacuously pass if Migration.operations is empty; add an explicit assertion that the number of migration operations equals the number of implicit fields to harden the test. In test_all_altered_fields_disable_db_index, assert len(Migration.operations) == len(self.migration_module.IMPLICIT_ROLE_FIELDS) (or == 38 to match test_migration_targets_correct_field_count) before iterating, so you ensure Migration.operations is populated and aligned with IMPLICIT_ROLE_FIELDS.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In
`@awx/main/tests/functional/migrations/test_0206_remove_implicit_role_field_indexes.py`:
- Around line 21-27: The test can vacuously pass if Migration.operations is
empty; add an explicit assertion that the number of migration operations equals
the number of implicit fields to harden the test. In
test_all_altered_fields_disable_db_index, assert len(Migration.operations) ==
len(self.migration_module.IMPLICIT_ROLE_FIELDS) (or == 38 to match
test_migration_targets_correct_field_count) before iterating, so you ensure
Migration.operations is populated and aligned with IMPLICIT_ROLE_FIELDS.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Enterprise
Run ID: b767c317-b8fb-457a-bf24-fe076da7c310
📒 Files selected for processing (3)
awx/main/fields.pyawx/main/migrations/0206_remove_implicit_role_field_indexes.pyawx/main/tests/functional/migrations/test_0206_remove_implicit_role_field_indexes.py
SUMMARY
Sets
db_index=FalseonImplicitRoleFieldand adds migration 0206 with 38AlterFieldoperations to drop stale FK indexes on legacy*_role_idcolumns from model tables. These indexes have zero scans after the DAB RBAC migration (0192+) and add unnecessary write/VACUUM overhead.This approach keeps Django's migration state in sync with the actual database schema (as suggested in review), rather than using raw
RunSQLwhich would create a mismatch between the field definition and the database.Related #16391
ISSUE TYPE
COMPONENT NAME
STEPS TO REPRODUCE AND EXTRA INFO
Before migration — 38 indexes with zero scans:
Affected tables: main_organization (13), main_inventory (5), main_workflowjobtemplate (4), main_project (4), main_credential (3), main_team (3), main_jobtemplate (3), main_instancegroup (3).
After migration: all 38 indexes removed, write performance and VACUUM improved.
Summary by CodeRabbit