Skip to content

New Parser: ru.alexandermalikov.protectednotes - #1888

Open
jensstigaard wants to merge 12 commits into
abrignoni:mainfrom
jensstigaard:parser/privateNotepad
Open

jensstigaard wants to merge 12 commits into
abrignoni:mainfrom
jensstigaard:parser/privateNotepad

Conversation

@jensstigaard

Copy link
Copy Markdown

What this changes

Parsing app ru.alexandermalikov.protectednotes artifacts: Notes, settings, passwords, attachments.

For a new or changed artifact

  • Ran the tool against a real extraction and confirmed the row counts, not just that it imports.
  • Ran python admin/scripts/check_artifact_output.py <report folder> on that report and fixed or documented every finding. An empty or constant column is often a real result (no group chats, coarse location denied); the fix for those is a sentence in the artifact's notes naming the column and saying it was empty or uniform, which is also what stops the checker reporting it.
  • Checked it against a second app data directory where the platform provides one (a second user, account, or container). --compare <multi-container report> reads the scaling for you: it should be exactly double.
  • notes, description and sample_data say only what the data shows, and the numbers were re-derived from the finished run.

Anything reviewers should know

Includes unit test.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Test data is now included for every changed artifact module. Thank you!

@github-actions github-actions Bot added the needs-test-data Artifact PR without test data for the changed modules label Nov 6, 2026
Jens Grønhøj Stigaard and others added 4 commits November 6, 2026 13:49
Add four artifacts for Private Notepad - safe notes
(ru.alexandermalikov.protectednotes). They decrypt the app's notes,
note attachments, stored passwords and lock screen settings.

The data key is unwrapped with the app's built-in static key. It comes
from the dek column of service_data in notes.db or from the
encryption_key preference, and the candidate that decrypts the
container's own data is used. Fields that do not decrypt are reported
as stored and logged.

Attachments are matched to files by id under the app's folders for the
same Android user. Ids that match no file are reported as File not
found, not as missing evidence. Files that decrypt to a recognised
media type but are not named by any note are listed too.

Protection type labels, date column semantics and the password hash
algorithm are marked as unverified in the notes. The module was tested
only on synthetic data built with the scheme of app versions 6.9.1
and 6.11.1.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Add a unittest suite for the Private Notepad artifacts. Each test
writes a real notes.db, preferences file and attachment files to a
temporary folder, built by running the app's encryption chain in
reverse: a data key wrapped under the static key, and text and files
encrypted under the data key.

The static key is held in the fixture as a known answer rather than
taken from get_static_key(), so a wrong key cannot pass by both sides
agreeing on it.

The schema is the one the parser queries, not one observed on a
device, so these tests show the parser handles this scheme and say
nothing about what an installed app writes.

Covered: key resolution order and fallback, fields and attachments
that do not decrypt, per-user and mirrored storage folders, shared
storage matching, read-only handling of the evidence, and the
artifact path globs.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Add columns to Private Notepad - App Settings for the fake_password_hash
preference and for the first row of the service table in notes.db. The
service values sit beside the preference values rather than replacing
them: in the code of the 6.9.1 APK (versionCode 108) that table is only
read on an upgrade, when its values move to the preferences, and nothing
naming it writes to it, so a value there can be older than the
preferences. They also give a row when the preferences file is missing
or unreadable, and they are read without needing the data key.

Update the notes in all four artifacts. From reading the APK's code, not
from a device with a lock set: the defaults of absent keys, the
protection type labels, and the hash as base64 of a SHA-1 digest to
which the hash function adds no salt. The notes also state when each
blank column is blank, so check_artifact_output.py has nothing to report.

Replace the claim of synthetic-only testing with what ran: one real
extraction of app 6.9.1 on Android 14, giving 13 notes, 1 settings row
and no attachments or passwords. A real attachment, password or set lock
was not exercised, nor was service_data.dek or app version 6.11.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the pattern that took everything under the app's folder with the
locations the code of the 6.9.1 APK (versionCode 108) uses: notes.db and
its sidecars, the preferences file, files/images and files/recordings.
On a real app folder this reads 3 files rather than 175. The patterns
stay anchored on the package rather than on data/data, so a second
Android user and an extraction holding only data/user/0 are still read.

Match an attachment by its exact file name in the folder for its column,
as that code names them, and drop the fallback to a name containing the
id, which could join a note to the wrong file. Android/data is no longer
read: a copy matched there is rejected because it names no Android user.
Every file in the two folders that no note names now gets a row, blank
under Attachment when it does not decrypt, with its type and name.

Add Private Notepad - Intruder Photos for files/intruder_photos. In that
code the photo bytes are written with no encryption step and the file is
named from the time it is created, so the artifact uses no key and
reports File Name Time from the name. What prompts a capture and who
appears in a photo are not established, and the notes say so.

No real attachment or intruder photo has been read: the one real
extraction has none. Both paths ran only on synthetic data and
placeholder files, and other app versions were not checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jensstigaard
jensstigaard force-pushed the parser/privateNotepad branch from e8adec4 to d8f7896 Compare November 6, 2026 12:49
@abrignoni

Copy link
Copy Markdown
Owner

Thanks Jens, this is a careful module. The notes say clearly what was tested on real data and what was only synthetic, and the CI checks all pass. A few things before merge:

  1. Fixture. Please add one for the module. From your extraction:

    python admin/test/scripts/make_test_data.py privateNotepad --case <case_number> --input <extraction.zip>
    

    That writes admin/test/cases/testdata.privateNotepad.json and a zip per artifact under admin/test/cases/data/privateNotepad/. If the extraction can't be shared, two options work: overwrite the personal values in the real notes.db and prefs file and keep everything else as the app wrote it, or install the app on a test device, create a few notes, a password entry, an attachment and a lock, and extract that. A scripted session would also give real data for passwords, attachments and intruder photos, which today are only tested on synthetic data.

  2. Licence header. The zlib notice says the file is an altered version of other software, but it doesn't say which. Who wrote the original, and where is it? We need to name it so it can be credited. If you wrote all of it yourself, the header can go, since the repo is MIT.

  3. Key check. resolve_key tests each candidate key against one field, the first non-empty title, body or password. If that one value is damaged, the right key is rejected and the whole container gives no rows. Could it try a few fields, or fall back to the only key that unwraps and report the fields that fail as stored?

  4. Notes length. The paragraph on the data key and decryption is repeated in four artifacts, and in App Settings it points to Key Source in App Settings, its own artifact. One sentence in each artifact, with the full account in Notes, would be easier to read in the report.

  5. Small ones:

    • hash_as_hex calls b64decode without validate=True, so stray characters are dropped quietly and the hex can be wrong instead of falling back to the stored value.
    • A last_app_usage_time that isn't a number ends up as text in a datetime column.
    • Notes and Passwords have no Android User column, while Settings and Intruder Photos do. Adding it would make them match.

Thanks again for the work on this one.

Jens Grønhøj Stigaard and others added 3 commits November 9, 2026 09:05
Move the synthetic data builders (key constants, encryption helpers,
and the Extraction class) out of test_private_notepad.py into
private_notepad_fixture.py. The test module now imports them and
subclasses Extraction only to add the seeker context stand-in.

The module can also be run as a script. It writes one persistent
extraction that exercises every artifact (two Android users, notes,
attachments, passwords, settings, intruder photos), so ALEAPP can be run
over a folder with -t fs without a real device image. As with the tests,
the data proves the parser handles this scheme and says nothing about
what an installed app writes.

Extraction.database() now takes a service argument, replacing the
test's own helper that reopened the database to add the table.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The module no longer carries the zlib-style license notice, and the
artifact author fields now list only the human author. Per project
convention the author string stays unversioned and reaches the HTML
report and LAVA manifest.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Add the casejens fixture zips and the testdata.privateNotepad.json
manifest covering private_notepad_notes, attachments, intruder_photos,
passwords and settings. The manifest was generated against the current
paths, so file counts match the narrowed globs.

expected_output is still empty for every artifact, so these cases
exercise path matching only until real rows are recorded.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot removed the needs-test-data Artifact PR without test data for the changed modules label Nov 9, 2026
Jens Grønhøj Stigaard and others added 2 commits November 9, 2026 09:20
A damaged first stored field made the old check reject the right key,
because only one sample was tested. The key was then dropped and the
container gave no rows.

Test each candidate key that unwraps on up to 8 stored text fields of
the notes and passwords tables, and use the one that decrypts the most,
service_data.dek first on a tie. When no key decrypts a tested field,
use the first anyway so rows are reported with their fields as stored
instead of being left out. Key Source now says how many tested fields
decrypted when it was not all of them.

Shorten the notes of the four other artifacts to point at Private
Notepad - Notes for the shared key description, and add tests and an
Android user 11 case to the fixture for the damaged-field scenario.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Add an Android User column to the notes, attachments and passwords
artifacts so rows from different user containers can be told apart, as
the settings artifact already does.

Decode Password Hash with validate=True so a value with stray
characters is reported as stored instead of being silently decoded to
a different hash. A last_app_usage_time that is not a valid time in
milliseconds, including one that overflows, is now left blank and its
stored value logged, rather than showing the raw string in a datetime
column.

Tests are updated for the new column and cover both new behaviours.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@jensstigaard

Copy link
Copy Markdown
Author

Hi Alexis

Thanks for the review statement.

For reference, my answers to the stated bullets.

  1. Commit 27575f7 implements a separation of logic enabling smoothly generation of test data.
    Commit 5d6c09f adds a generated zip containing test data for the parser.

  2. I have cleaned up the parser script, removing the license header, which were unclear.
    In cooperation with a colleague we originally made a script in 2024, parsing the app for XRY/XAMN. The parser script in this PR is basically just a port of the code from the XRY script authored by me and my collegaue. My colleague is not interested in figuring in the code/repo as an author by the way...

  3. Handled in f732589

  4. Handled in f732589

  5. Handled in 1c6b34a

I have run latest code and tested against repo test data + real data, and it returns the expected number of artifacts.

@abrignoni

Copy link
Copy Markdown
Owner

Thanks Jens, the changes look good and the key selection is much safer now. Two CI checks are red, both small:

  1. Test Cases: the case zips are in, but there is no recorded snapshot, so all five units report NO_BASELINE. Record them with:

    python admin/test/scripts/test_module.py privateNotepad -a all -c casejens
    

    and commit the JSON files it writes under admin/test/results/privateNotepad/.

  2. Python Lint Check: pylint flags unsubscriptable-object on best[0] in resolve_key, because best starts as None. Keeping the count in its own variable (for example best_count = -1) clears it.

The synthetic fixture is fine as a regression case. Your builder says it plainly, that it uses the schema the parser queries and not one seen on a device, so a short note in the Notes artifact saying the committed test data is synthetic would round it out.

On the licence: thanks for explaining. Since the XRY script was written with a colleague, please confirm you are both fine with this port going out under ALEAPP's MIT licence.

Jens Grønhøj Stigaard and others added 2 commits November 11, 2026 18:22
Track the best decrypted count in a separate best_count variable
instead of packing it into the (count, source, key) tuple. Behaviour is
unchanged: the first candidate with the highest count still wins.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@jensstigaard

Copy link
Copy Markdown
Author

1+2 are now fixed with the latest commits.

We confirm the acceptance of MIT licensing when contributing.

@abrignoni abrignoni closed this Nov 12, 2026
@abrignoni abrignoni reopened this Nov 12, 2026
The snapshots were recorded on Windows, so their paths used backslashes.
The test harness compares paths as recorded on Linux and macOS and only
folds backslashes on Windows runs, so all five units failed on the Linux
Test Cases job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants