JetBrains.dotMemory.Model

Represents memory block containing objects organized as skip list. See https://en.wikipedia.org/wiki/Skip_list

Address of first byte of memory block.

Number of levels.

Return all objects in memory block.

Searches requested objects in this memory block and puts found objects with assigned number to numberedObjs dictionary.

The MUST BE ordered by Address.

Inserts single object into this memory block. Objects are ascending ordered by its address.
Complexity: O(log n), n - number of objects in this block

IMPORTANT! This method expects that all objects have unique addresses. Inserting several objects with the same address has undefined behavior.

Inserts multiple objects if they are absent in this memory block. Each inserted object takes number provided by numberGetter functor. All objects are put to numberedObjs dictionary with appropriate numbers.
Complexity: O(k*log k + n), n - number of objects in this block, k - number of objects in inserted sequence

The MUST BE ordered by Address.

Shift memory block on bytes

true if shift positive and false otherwise

Add to the end of current block.

IMPORTANT! After concatenation rightBlock might contain some junk (not necessary empty). Start of right block should be >= end of left block

Splits memory block into two parts. Second part will be

IMPORTANT! result is being overwritten during split operation. Split address is beginning of second memoryBlock Tail part wil be located here Should be at least Second block

Skipping strategy defines objects distribution per levels in skip list.

Returns level for newly inserted node. Level must be in range [0, levelsCount).

Random skipping: object propagates to next level with probability 0.25

ObjectNumber in original snapshot

Addresses of node on different levels

[k] next element in skipped list for level k

Delete memory block from to

Source memory block Beginning of extracted block End of extracted block Extracted block will be here should be greaterOrEqual

Insert in proper place

Temporary, helper class

Converts plain to n-dimensional index. If < 0 then null is returned (see remarks).

There are several special cases when outgoing array references have ArrayIndex < 0:
* array type is a type generated in runtime, then first reference points to internal LoaderAllocator instance; * (ask M.Pilin)

Builds compact call tree: grouped per-type on each level.

MUST have exactly the same layout as

MUST have exactly the same layout as

MUST have exactly the same layout as

The overriden implementation of specialized for ClrDump workspaces.

works in a separate process log file name works in a separate process log file name

ClrMD loads dbgeng, which in turn tries to load dbghelp, but instead of loading the system one sometimes it can get a wrong copy from elsewhere. The workaround manually loads dbgeng.dll from the Windows system directory and forces it to look for its dependencies only there. See https://github.com/Microsoft/clrmd/issues/29

see https://msdn.microsoft.com/en-us/library/windows/desktop/ms680519(v=vs.85).aspx

Mainly name-based id, with some additional restrictions on relevant metadata index and token (when we have such info)

metadata index or 0 if there is no metadata for the module

The overriden implementation of specialized for ClrDump workspaces.

SequentialIdMapper

Profiler writes method ids (aka fuids) in packed format. It's either MetadataIndex and Token packed into a single uint or an index in an auxiliary section (in case if it's not possible to pack)

Writes predefined method ids relevant to profiler.

Generally, removes interface name for explicitly implemented methods.

Method => Method
IExplicitInterface.Method => Method
<IExplicitInterface.Method>x_lambda => <Method>x_lambda
.ctor => .ctor
.cctor => .cctor
<.ctor>x_lambda => <.ctor>x_lambda
<.cctor>x_lambda => <.cctor>x_lambda
<>compiler_name => <>compiler_name

Auxiliary table for FUIDs which can not be encoded in-place.

All FUIDs appeared in the call trees. Be aware, might contain the same FUID several times.

This entity is a part of .
BE AWARE: RawFuidsPerCallTree is flushed only when call tree rotated.

All FUIDs appeared in the call trees. Be aware, might contain the same FUID several times.

This entity reads FUIDs from call tree nodes.

IMPORTANT! The returned stream must be used instantly. Opening blob streams on several Entry-s at the same time is not allowed.

Reads finalized objects per each gc in reverse order, i.e. the first one is gc occurred during snapshot, second - gc just before snapshot and so on. If no finalized objects on particular gc then empty sub-sequence corresponds to this gc.

finalized objects per each gc in reverse order

Unique asc- sorted .

Unique asc- sorted by number root objects.

The index-part of adjacency list containing roots per each object.

Finds pair of different retention paths for each object in graph.

The first path is the shortest path to root, the second one is just another path defined by heuristic to be 'different' and to be 'short'.

IMPORTANT: The algorithm isn't mathematically strict but provides better performance comparing to Suuarble-Torjan algorithm implemented in .

This is the implementation of Suuarble-Torjan algorithm (https://www.eecs.yorku.ca/course_archive/2007-08/F/6590/Notes/surballe_alg.pdf) which looking for shortest paths pair from meta-root to each object in the graph.

DominatorsTree[n] is dominator of n-th object

The shortest path to root tree as result of breadth-first-traverse of direct graph.
RootPathTree[n] is a parent of n-th object in shortest retention path

Reverse graph reference index for .

reverseGraph[RootPathTreeReferenceIndex[n]] == RootPathTree[n]
reverseGraph.GetDetails(RootPathTreeReferenceIndex[n]) // array index/field in parent which references n-th object

In case of refactoring run VxShortestPairsBuilderTest.GoldTest

In case of refactoring run VxReverseGraphBuilderTest.GoldTest

The shortest path to root tree as result of breadth-first-traverse of direct graph.
RootPathTree[n] is a parent of n-th object in shortest retention path

Reverse graph reference index for .

reverseGraph[RootPathTreeReferenceIndex[n]] == RootPathTree[n]
reverseGraph.GetDetails(RootPathTreeReferenceIndex[n]) // array index/field in parent which references n-th object

document me

document me

Guarantees equal ids for equal strings

Required to - perform merge of duplicate entries in class section - match entries obtained from metadata section (e.g. information about interfaces) with existing entries from class section Initialized in 3 runs (in order to handle cyclic references): - first run just creates all instances - second run sets generic arguments - third run performs initialization: cycles detection and hashcode calculation

Objects referenced via static fields. Might contain duplicates! Synchronized by index with .

Static reference details. Synchronized by index with .

Builds compact dominators tree: grouped per-type on each level.

The story of optimization: https://habr.com/ru/company/JetBrains/blog/574722/

MUST have exactly the same layout as

Dominator tree node: represents compacted set of objects of single type (just type and count stored).

Compact means object instances grouped (compacted) by type.

Type of objects represented by this node.

Number of objects with type ObjectsType which dominates (exclusively retains) Children.

NOTE! If compact dominator tree is built on some object subset (not all objects in snapshot) then ObjectsCount might include objects which aren't in analyzed subset but dominates someone from it.

Number of objects retained by all objects of this node, including this.ObjectsCount.

Number of bytes retained by all objects of this node including this node's objects.

Whether at least one object (associated with node) contained in analyzed object set?

The False value means 'node is just a part of domination path and no objects of appropriate type in analyzed set'.

Compact nodes for objects dominated by all objects of this node.

Compact node for object which dominates all objects of this node.

Can return multiple objects if field is a collection of ValueTypes

The helper matcher for ambiguous cases: A.B.Hz/*namespace or type*/.Hz/*namespace, or type, or field, or method*/

IMPORTANT! The matcher is stateful and thread UNSAFE.

Filter "text" as programming language identifier, method, type, and so on. Using CamelHumps for matching.

Filter "text" using system .NET regexp

The specified pattern shouldn't be matched with generic arguments.

The specified pattern should be matched with namespace parts only.

The specified pattern should be matched with type name only.

The specified pattern should be matched with method name only.

The array types should be excluded from filter result.

The only value types should be included into filter result.

Matches the first sub-sequence defined by within traversing from left to right and respecting wildcards (any number of any parts).

Matches the last sub-sequence defined by within traversing from right to left and respecting wildcards (any number of any parts).

Throws if is invalid regular expression

Query: 'awesome' (just single word)
Searches in: namespace, type name, type generic arguments (recursively), method name, method generic arguments (recursively).

Field/json property names should correspond to declared in Rider/Vs

The count of "empty" ports we should handle before stop to look for Rider.

Several solutions opened in the Rider of one version produce only one OS process

For now it is implemented as copy past of V2 for the sake of the velocity Next time it could be rewritten to reuse some common logic, at least exception handling as the most complicated part

After any exception (e.g. timeout) wcf client is set to the Failed stated and can't be used anymore, that is why we pass clientFactory along with client

Represents thread safe map (id -> scope) of all snapshot scopes in the profiling session.

This class is a thread safe wrapper over IReadonlyCollectionEvents<ISnapshotScope> with O(1) access by id. The direct enumerating of IReadonlyCollectionEvents<ISnapshotScope> to find scope by id isn't thread safe because new scope might be added concurrently during enumeration.

Helps to avoid duplication of array rank arrays and array postfix strings

Loader for cached metadata which were obtained as result of raw-meta processing from profiler core.

Loader for dumped (provided as is) metadata (e.g. ClrDump, JavaDump).

Base class for readonly entities capable of being exposed to WPF. should be implemented in order to avoid memory leaks.

Persist-able fixed size field entry.

Gets streamable metadata instantly if it already includes all data currently available in streaming.

Refines streamable metadata using all data currently available in streaming.

Refines streamable metadata using fields info from specified snapshot.

Implements the logic of reading saved metadata from the disk and passing it to Metadata on the disk is stored in the final dotMemory format.

Persist-able fixed size module entry.

Persist-able fixed size namespace entry.

Replaces all special symbols in string with their human friendly names using or return original string.

It's not static for optimization reasons - to not create on each call but reuse the one instance.

Returns a human friendly name of the special symbol

Returns a list of indicies of special symbols in string if any

Implementation of keeping all the data in the memory

Validates strings order to early detect issues like DMRY-9381.

Specifies metastamp till which metadata should be processed. NOTE: Used for test purpose only! See StreamableMetadataReplayTest.

Gets the persistent cache key for next patch.

IMPORTANT! If you need to increment metadata cache revision then most probably you need to increment whole cache version (WorkspaceStateFolder.CacheFolder).
This is because the TypeId-s produced by metadata processing depends on requests order. The processing in other order or even full processing at once will generates another TypeId-s which will not match TypeId-s stored by other entities. So when metadata re-calculated the other entities also should be re-calculated.

Logs test data used by StreamableMetadataReplayTest.

Special implementation of for

Persist-able fixed size type map entry.

Generates set of objects having given types at provided domination path.

For example:
Let A dominates B; B dominates C1, D, E; D dominates C2.
For path 'A > B' and types C, D the set containing C1 and D will be generated.

Generates set of objects at provided domination path.

For example:
Let A dominates B1, B2, and C; C dominates B3.
For path 'A > B' the set containing B1 and B2 will be generated.

Note, neither A, nor C, nor B3 is included.

Is not tested with dotMemory, only with dotMemory Unit

Generates set of objects which are retained by objects having given type at domination path.

For example:
Let A dominates B, B dominates C, D and E, D dominates F
For path 'A > B' and types 'D, E' the set containing D, E and F will be generated.

Generates set of objects which are retained via provided domination path.

For example:
Let A dominates B, B dominates C and D.
For path 'A > B' the set containing B, C and D will be generated.
Note: objects at end of path also included.

Implementation of which creates instances of as

ctor and for details

The RawCallTreeForest-based implementation without internal caching.

If some new features to the snapshot format are only added and no one of existing are changed, increase number. It means "new" snapshot can be opened by previous version of dotMemory. But old snapshots can't be opened by the new version of dotMemory. If the snapshot format is changed increase and possible If old snapshot can be opened by new version of dotMemory (usually using different data readers for the old and the new format) leave unchanged. If snapshots are totally incompatible make equals to

Indicates minimum can be opened by "this" dotMemory Is used during opening a snapshot to determine can it be opened by "this" dotMemory

The "main" version of the snapshot. Written to snapshot data during conversion by "this" dotMemory

The "feature compatibility" version of the snapshot. Written to snapshot data during conversion by "this" dotMemory Also it is used during opening a snapshot to determine can it be opened by "this" dotMemory Snapshot which have greater build number than "this" dotMemory can be opened

Is not used for determine compatibility, used only for representation

Constructs snapshot writer as NEW multi-file snapshot.

Constructs snapshot writer as APPENDABLE multi-file snapshot.

This method is used in order to reuse one or several files from the multi file snapshot. It renames these files thus breaks the original snapshot. Write information about all sections located in these files to own index file.

Collection of descriptors of section which should be reused. All files where these sections are located joined to this snapshot. Functor returns the source snapshot file path by source snapshot file index. Use this method carefully, if you call it twice passing sections from the same file, it crashes due to old file is already renamed.

Sequential number of section of this

In fact, it's GC number for GC dependent sections, sequential number for Allocation CallTree section, and for widesnapshot sections

Describes profiling session in the workspace index file

Relative (against this profiling session) path to streaming storage.

Describes memory snapshot in the workspace index file

Used by to persist index of stored data. Data "name" is used as indexed key

Different versions of dotMemory can store data in defferent format, so this field contains collection of [formatVersion, physicalStorageName]

Buildable version of , MUST have the same fields layout.

Pre-merges raw allocation statistic into blocks approximately by 256 GCs.

The statistic might be pre-merged in any order. The previous merges are accumulated and stored in session-level cache. Ensure correct registration: IPersistentStorage should be bound to session-level cache folder.

Merges raw call trees into single tree.

The trees might be merged in any order. The previous merges are accumulated and stored in session-level cache. Ensure correct registration: IPersistentStorage should be bound to session-level cache folder.

Mutable version of .

IMPORTANT! The layout MUST BE exactly the same as .

Merges allocation statistic in given GC-range using pre-merged blocks.

Used for limited read of obsolete dotMemoryUnit 2.x workspace.

Reads only workspace meta info without sessions and snapshots, just to show incompatibility warning.

Is used to specify what unit should be inject into consumer if it's impossible to distinguish an unit by it's type. See usages of fields for details.

Reads workspace from raw snapshot either zipped or not and returns fictive workspace index to redirect to portable version for backward compatibility.

For zipped raw snapshot workspacePath should be a zip file path.
For unzipped one workspacePath should be a raw snapshot index file path (*.dmrs).

Sweeps out broken and temporary entities. Might be called several times during lifetime.

Removes unnecessary all entities on lifetime termination.

Used for limited read of obsolete dotMemoryUnit 2.x workspace.

Reads only workspace meta info without sessions and snapshots, just to show incompatibility warning.

Initializes new ZippedJsonWorkspaceProcessor.

If compatibilityMode true then stub workspace.idx file is generated additionally. The stub workspace.idx contains only ProductName, ProductVersion and WorkspaceVersion. This is necessary for nice error message when new workspace is opened in previous DM versions.

Enumerates types with constructed method tables in all modules.