JetBrains.dotMemory.Core

DO NOT use this class directly (only for setting activity tracking up) use ActivityTracker from the corresponding namespace This class is used to provide static access to the real implementation from different layers of the app

Call this method if dotMemory performs active profiling rather then opening saved workspace or dump Some Events are reported only during active profiling session due to do not duplicate data such as snapshot's "TotalObjectsCount"

The semantic wrapper over Armature's arbitrary token.

Extensions to support long-dependency concept.

The long-dependency concept was introduced to simplify and unify entities resolution for view-models. The most VM-s requires some set of entities to start working (in other words VM completely useless until entities have resolved). Long-dependency mechanism provides intermediate stub-vm with progress while resolving required entities, after completion it switch to actual VM. See IHavingPrerequisitesVm. The VM just declares its required entities with [LongDependency] attribute.

The most work related to long-dependency resolution performed in BuildWithLongDependencies method.

The related stuff:
*
*
*
*
* IHavingPrerequisitesVm

Some examples:
* Dominators.DominatorTreeCompoundVm
* Traffic.v2.AllocatedTypesPairedVm

Registers for accessing long-dependency .

IMPORTANT! This is disputable solution, DO NOT use it in new code, if absolutely necessary.

Adds another rule to the rules list. Modifies the original list. Fluent.

We need to exclude dotMemory itself from profiling in case of "Profile all .net processes" mode

Creates instance of via .
If `T` is not registered then is thrown.
If `T` is registered as null-instance then InvalidOperationException is thrown.
In case when null is valid value use instead.

See

Returns blob content as byte array.
NOTE. This method for backward compatibility, it isn't recommended for new code.

Represents basic entity: adjacency list stored on the disk with random read access.

IMPORTANT! Disk-entities aren't thread safe - you must not access the same instance concurrently.

Reads items at given range.

DO NOT USE returned reader after consequent call to ReadItems.

    
            var range = adjacencyList.ReadSubRange(currentIndex); 
            var items = adjacencyList.ReadItems(subRange);
            while (items.Next(out var item))
            {
              // ...
            }

The shortcut for ReadItems(this.ReadSubRange(currentIndex)).


            var items = adjacencyList.ReadItems(currentIndex);
            while (items.Next(out var item))
            {
              // ...
            }

Represents basic entity: array stored on the disk with random read-access only.

IMPORTANT! Disk-entities aren't thread safe - you must not access the same instance concurrently.

The empty instance of disk array.

Implemented via property to prevent possible exception on static-initialization phase.

The fluent-style API for construction.

Constructs from .

Specifies externally provided length (in items) of array.

Specifies buffer length (in items) used for reading.

Specifies number of items which should be pre-fetched by default after seek.

The default value is 1.
The value 0 means no pre-fetching performed after seek but first read operation will load entire buffer.

Reads array length from header as Int32.

Reads array length from header as Int64.

Read size of HeapPtr from header as Byte.

Reads HeapPtr either UInt32 or UInt64 according to size of HeapPtr read previously.

Reads Byte from header.

Reads UInt16 from header.

Reads UInt32 from header.

Reads Int32 from header.

Reads Int64 from header.

Reads UInt64 from header.

Skips specified number of bytes of header.

Set-up item serializer.

Builds .

Sets write position to the end of region and returns current array length (items count).

Array with explicitly specified length can not be appended.

The fluent-style API for construction.

Constructs from .

Constructs from - creates (if needed) and .

Set-up item serializer.

Writes length (in items) of array.

Array with explicitly specified length can not be appended.

Writes length (in items) as 32-bit int.

Writes pointer size.

Builds .

Represents basic entity: optimized byte-array stored on the disk with random read access.

IMPORTANT! Disk-entities aren't thread safe - you must not access the same instance concurrently.

Represents non-virtual graph data structure stored on disk.

Implemented via .

Reads range of indices where references from are stored.

If only references count is needed then you might read just a range without followed ReadReferences.
Also it is allowed to narrow a range before passing to ReadReferences. E.g. it might be useful for paging.

Reads references at given range.

DO NOT USE returned reader after consequent call to ReadReferences.


            var range = diskGraph.ReadSubRange(currentNode);
            // narrow range if necessary
            var references = diskGraph.ReadReferences(range);
            while (references.Next(out var reference))
            {
              // ...
            }

The shortcut for ReadReferences(this.ReadSubRange(currentNode)).

DO NOT USE returned reader after consequent call to ReadReferences.

    
            var references = diskGraph.ReadReferences(currentNode);
            while (references.Next(out var reference))
            {
              // ...
            }

Represents non-virtual graph stored on disk with additional data associated with the edge.

Reads reference details at specified range.

Reads reference details from given node.

Reads reference targets and details at specified range.

Reads reference targets and details from given node.

Provides fluent-style API for registering entities accessed via .

There are several types of entities:

* in-memory JetArray-based
** supports memory time-based caching (hold for some time after releasing all usages)
** supports disk caching (generalized or custom)
** suitable for both value- and reference-type elements, but...
** ...RECOMMENDED for reference-type elements only
** ...for value-type elements the VxArray-based way is HIGHLY RECOMMENDED
** provides immutable data, so thread-safe without any additional synchronization
** provides the same instance for all requests

* DiskArray-based aka readers
** random access to immutable data stored on disk
** provides new instance for each request, returned instance must be used in single thread only because contains current read position
** supports access to both raw- and calculated- data
** used when access to small pieces of data is needed, but...
** ...ISN'T RECOMMENDED for new entities, the VxArray provides better experience in almost all cases

* memory mapped VxArray-based
** the VxArray has built over memory-mapped file technique:
operates over raw memory, provides highly efficient IO operations and supports partial loading
** supports disk caching by definition
** supports memory caching by design (OS leaves released pages in file cache while amount of physical memory permits)
** supports effective access to small pieces of data by design (due to per-page loading)
** supports value-type elements ONLY
** HIGHLY RECOMMENDED for all cases whenever possible

* tuple
** introduced for cases when calculation algorithm produces several entities at once, but then each entity should be accessed separately
** each entity in tuple stored in own disk-cache slot

* tuple of VxArray-based
** solves the same case as tuple above but has simpler implementation due to memory-mapped file technique

* compound
** composes entity from several sub-entities
E.g. object graph composed from indices and references and/or details.
The interesting moment here that indices are taken directly from snapshot, but references and details are taken from calculation cache. TODO: consider rename to 'EntityRegistrar'

Registers entity-ref to in-memory entity as . The entity instance is shared by consumers.

The shortcut for Register<TEntity,NoTag>

The shortcut for RegisterReader<TDiskEntity,NoTag>

Registers entity-ref to tuple of entities as . The tuple is shared by consumers.

Some algorithms produce several entities at once, i.e. tuple of entities.
The tuple just an internal artifact of algorithm, it should not be used by any upper-level logic.
Instead, each entity from tuple processed independently with its own cache-chain. So, INTENTIONALLY DOESN'T HAVE the methods:
* With(PreresolutionOrder)
* WithMemoryCache
* WithDiskCache
See ReverseGraphTuple registration for example.

The shortcut for RegisterTuple<TTuple,NoTag>

Registers entity-ref to VxEntity - entity based on VxArray and memory mapped technique. The entity instance is shared by consumers.

Registers custom entity-ref.

Registers entity-ref to long dependency registration. The long dependency itself must be registered separately. The entity instance sharing is controlled by long dependency declaration.

Adds dependency of currently tuned entity to the entity tuple. When tuple will be resolved then all dependent entities get `PreresolveAsync` called (this provides facility to store to disk cache parts of tuple as separate entities). The lifetime of entity tuple will not be terminated until all pre-resolution tasks are finished.

Registration tuner for tuple of entities.

INTENTIONALLY DOESN'T HAVE the methods:
* With(PreresolutionOrder)
* WithMemoryCache
* WithDiskCache
See for details.

The entity is evaluated and already in memory.

The entity is evaluated and stored on the disk.

The entity is evaluating (task is scheduled or running but result not yet available).

The entity isn't evaluated and task isn't scheduled yet.

The entity isn't available (no data were written)

Generalized depth-first-search algorithm.

IMPORTANT! If is a value type then performance will be the same as of specialized implementation (interface calls on value types are successfully inlined by compiler).

Generalized depth-first-search algorithm.

IMPORTANT! If is a value type then performance will be the same as of specialized implementation (interface calls on value types are successfully inlined by compiler).

Generalized breadth-first-search algorithm.

IMPORTANT! If is a value type then performance will be the same as of specialized implementation (interface calls on value types are successfully inlined by compiler).

Generalized depth-first-search algorithm with callbacks.

IMPORTANT! If both and are a value types then performance will be the same as of specialized implementation (interface calls on value types are successfully inlined by compiler).

Specifies to collect spanning tree.

IMPORTANT! `SpanningTree` requires to be specified also.

Accessibility marks.

Spanning tree according to traverse order. The SpanningTree[n] is a parent of n-th node.

CAN BE NULL if appropriate collect option was not specified.

The reference index for appropriate node in SpanningTree. The SpanningTreeReferenceIndex[n] is an index of reference from parent to n-th node, i.e. graph[SpanningTreeReferenceIndex[n]] == (TNode)n

CAN BE NULL if appropriate collect option was not specified.

Nodes in pre-traverse order.

CAN BE NULL if appropriate collect option was not specified.

Nodes in post-traverse order.

CAN BE NULL if appropriate collect option was not specified.

Depth level.

CAN BE NULL if appropriate collect option was not specified.

Represents address inside managed heap of profiled application.

The global async-activity tracker for "heavy" tasks. Used to detect idle (neither cpu nor io payload) state to start background calculations.

Mix-in interface used for algorithm generalization (e.g. see )

Provides facility for persisting/restoring entity to/from disk storage.

Provides disk-entity to cached data.

The returned disk-entity is implicitly bound to room's lifetime.

The service performing background entities pre-resolution.

Service represents background activity and MUST BE bound to lifetime which will be terminated BEFORE scopes where pre-resolved entities registered.

Enqueues bunch of entity references for background pe-resolution.

Bunches are processed sequentially.
References from one bunch are processed in a single task with appropriate progress reporting, used as part of task title which is displayed to user.

Represents asynchronously resolved reference to entity (part of model, service, etc).

The implementations of this interface is expected to be thread safe.

See .

Tries to resolve entity synchronously: true if entity was in memory already and false otherwise.

Take the following in account:
* method doesn't start any async activity;
* entity instance might be either independent or shared (the last one is expected to be thread safe);
* produced entity directly or indirectly bound to (shared entity bound indirectly).

Mostly used within resolvers chain to reduce number of 'dummy' tasks.

Resolves entity in asynchronous way.

Take the following in account:
* method might return already completed task;
* task might produce either new independent entity instance or shared instance (the last one is expected to be thread safe);
* produced entity directly or indirectly bound to (shared entity bound indirectly);
* task itself is not bound to (lifetime termination doesn't cancel a task).

The shortcut for IEntityRef<TEntity,NoTag>.

Gets entity synchronously. Throws exception if reference can't be resolved synchronously.

IMPORTANT! Use this method for raw entities only. Only raw entities are guaranteed to be resolved synchronously.

Returns true if entity is available in current profile configuration and false otherwise.

The convenient shortcut to resolve entity reference.

Represents step in entity reference resolution pipeline.

Tries to resolves entity without potentially long-running operations.

Extracts single item (entity) from tuple of entities.

Used when some algorithm produces a set of entities at once (e.g. ReverseGraphBuilder).

Represents resolver of disk-entity which reads directly from disk cache.

The helper resolver which notifies tuple co-entities to be stored in disk cache.

Alternative implementation of which do not share entity instances, i.e. it returns new instance on each request. Might resolve entity either synchronously (returns finished task with result) or asynchronously depending of source's resolution state.

The main implementation of .

The empty implementation of .

The implementation for .

The implementation for regular .

It is possible to create several buffers over the same stream region, but they aren't thread safe, i.e. the caller is responsible for synchronization.

The implementation for .

Constructs memory mapped buffer over MMS section data.

IMPORTANT! Use parameter only if absolutely necessary, because it allows data modification behind .

The implementation for regular .

Represents set of named stream regions associate with a single as a part of .

The persistent room instance must be accessed from a single thread only.

Whether room has any readable region or not.

Whether room has any writable region or not.

The whole room length in bytes.

Gets readable region by id. If this is newly created room then exception is thrown.

Returned region is bound to room's lifetime.

Tries to get readable region by id. Returns true if region with specified id exists and false otherwise.

The room itself must be readable, otherwise exception is thrown.
Returned region is bound to room's lifetime.

Gets writable region by id. If this room is in read-only mode then exception is thrown.

Returned region is bound to room's lifetime.

Commits changes.

Due to cross-process nature of persistent storage the changes must be explicitly committed.

Represents cross-process persistent (disk) storage.

The storage must remain consistent even with concurrent cross-process access.

Returns true if readable committed persistent room exists. If the room doesn't exist at all or it is currently writing then method returns false.

Tries to open existing (readable) persistent room by given .

The room is opened in shared read-only mode.
If the room doesn't exist at all or it is currently exclusively locked then method returns false.

Opens existing (readable) or creates new (writable) persistent room by given .

The existing room is opened in shared read-only mode.
The new room is created in exclusive write-only mode.
If requested room already opened for write then storage waits for write finish using ..

Typically, this method used for immutable data - once created it never changes.

Opens existing or creates new readable-writable persistent room by given .

The existing room is opened in exclusive read-write mode.
The new room is created in exclusive read-write mode.
If requested room already opened for write then storage waits for write finish using ..

Typically, this method used for mutable data - once created it might be updated from time to time.

Low-level interface for efficient unified reading from regular or from memory mapped storage.

The whole data stream length in bytes. IMPORTANT! Might change if underlying stream is appendable.

Maps readable region at specified stream position and given size into memory.

Maps at least number. If not enough data available then is thrown (even if underlying stream isn't closed yet).
Actually tries to map number. If this value less than zero then default buffer length is used. If value less than requiredBytes then just requiredBytes is mapped.
The points to pinned memory block and stays valid till next call to MapIntoMemory or this buffer's lifetime termination.
Returns actual number of mapped bytes (this number is always greater or equal to ).

Starts asynchronous entity pre-calculation.

If entity already in-memory or on-disk then completed task should be returned.
If entity is evaluating then either running task or new task might be returned (depending of implementation).
if entity not available then completed task should be returned.

Represents reference to object which will be disposed after specified period of time after last use.

Last access time.

Period of life after last use.

Tries to re-use object. If object still alive then method increments internal ref-count (to prevent dispose while object is in use) and returns true with assigned. Otherwise method returns false with = null.

Releases the referenced object by decrementing internal ref-count. If ref-count is 0 then object marked to be disposed after .

The service which provides and handles timed references.

Creates timed reference and registers it in internal time table.

IMPORTANT! The is terminated from background thread.

Low-level interface for efficient unified writing to regular or to memory mapped storage.

The committed data stream length in bytes.

Maps writable region at specified stream position and given size into memory.

Maps at least number.
The points to pinned memory block and stays valid till next call to MapIntoMemory or this buffer's lifetime termination.
Returns actual number of mapped bytes (this number is always greater or equal to ).

Commits (flushes to underlying stream) data written to the buffer.

Represents basic entity: jagged array (array of items with array of sub-items) stored in memory.

This class is mostly intended for legacy data structures, for new one consider .

Array length (in elements) as Int64.

[BackwardCompatibility] Array length (in elements) as Int32 with range check.

The helper struct used with and which contains either index of first sub-item in the complementary array (for in-memory jagged array) or stream position of sub-array (for on-disk jagged array).

Represents adjacency list data structure (non-virtual, loh-free).

Represents basic non-virtual chunked (loh-free) array implemented over .

Underlying storage capacity.

Actually, JetArray isn't dynamically expandable array, but due to optimizations underlying storage might contain some excess items: see JetArrayBuilder and .

The property used for test purpose only.

Array length (in elements) as Int64.

[BackwardCompatibility] Array length (in elements) as Int32 with range check.

Returns IReadOnlyList wrapper over JetArray.

BE AWARE: The access to items via interface is much slower, so use it for compatibility only.

Expands array (actually creates new one and performs shallow copy of original content) similar to .

DO NOT USE original array reference after expanding.

Trims excess space if fill factor of is greater than .

Acts similar to . DO NOT USE original array reference after trimming.

See

Returns mutable shallow copy of this array.

The modification of returned instance doesn't affect original array.

Returns immutable shallow copy of this array.

The original array stays mutable and its modification doesn't affect returned instance.

Returns immutable array re-using storage of this array.

The original array becomes sealed and its modification isn't allowed.

Expandable builder of JetArray. Should be used when array length is not known in advance.

Prepares this builder to be re-used.

IMPORTANT! Access instances returned by previously has undefined behaviour.

Returns instance of .

Doesn't allocate or copy any bytes, re-uses internal storage as is (without trimming excess space).
The builder becomes sealed and followed Append() is not allowed until Reuse() called.
IMPORTANT! After calling Reuse() access to previously returned instance has undefined behaviour.

Returns instance of .

Doesn't allocate or copy any bytes, re-uses internal storage as is (without trimming excess space).
The builder stays non-sealed and followed Append() is allowed.
IMPORTANT! After calling Reuse() access to previously returned instance has undefined behaviour.

Returns instance of .

Constructs JetArray using internal storage with trimmed excess space.
The builder becomes sealed and followed Append() is not allowed until Reuse() called.
IMPORTANT! After calling Reuse() internal storage will be re-allocated, access to previously returned instance is safe.

Returns instance of .

Finds and moves the number of max elements to the begin of array, then trims excess items and return trimmed array.

In-place stable (merge) sort. Complexity = O(N log N) Memory = O(N)

Sort is in-place, use it carefully if concurrent access to is possible.
Allocates additional chunked array of the same size (no LOH allocations).
The only estimated progress is reported, not necessary reaches 100%.

In-place stable (merge) sort. Complexity = O(N log N) Memory = O(N)

The non-virtual and optimized analogue of BinaryReader.

The non-virtual and optimized analogue of BinaryWriter.

Syntax sugar for creating for a file. Disposes passed stream on termination.

Represents non-virtual, loh-free graph data structure.

Implemented via .

Returns sub-index of first reference of given .

Returns range of sub-indices which is used to obtain references of given . For usage example see

Returns single reference at specified .


            var subRange = graph.GetSubRange(currentNode);
            for (var j = subRange.Begin; j < subRange.End; j++)
            {
              var referencedNode = graph[j];
              // ...
            }

Returns internal indices and references arrays.

Represents non-virtual, loh-free graph with additional data associated with the edge.

Returns single reference details at specified .


            var subRange = graph.GetSubRange(currentNode);
            for (var j = subRange.Begin; j < subRange.End; j++)
            {
              var referencedNode = graph[j];
              var referenceDetails = graph.GetDetails(j);
              // ...
            }

The space optimized hash map with only 2 bits overhead for each entry. Implements so called "open-addressing" hash-map. TODO: this implementation might be reduced to 1 bit overhead

BE AWARE! It is slower than Dictionary about 2 times. Use for really large maps only.

This is copy-pasted implementation of with removed all virtual calls. The CompactMap is slower than Dictionary about 3 times.

The non-virtual read-only stream region.

IMPORTANT! Using several regions over the same underlying stream lead to undefined behavior. But it is possible to open several streams with share-read over the same file.

!!! BEWARE !!! When an instance of this class is used with memory mapping machinery, this value will differ from the "Position" from the "Reader"'s point of view, because Reader prefetches data from the stream into its internal buffer.

!!! BEWARE !!! See documentation, and use it as a for this method with caution.

Provides facility to read/write struct from/to stream by generic but efficient way.

The required to be value type with [StructLayout(SequentialLayout)] attribute.

Serializes struct T by fields using Unsafe operations.

Deserializes struct T by fields using Unsafe operations.

Serializer for paired struct-s.

The fluent-style API for construction.

Constructs .

Constructs default .

Specifies to ignore particular field.

Specifies to skip given number of bytes followed after particular field.
Used for skipping removed field.

Specifies to treat as UInt32.

The type must contain a single Int64/UInt64 field.

Specifies to treat all fields of as UInt32.

The type must contain only Int64/UInt64 fields.

Specifies to treat particular field of as UInt32.

The type must contain field with name and Int64/UInt64 type.

Specifies to treat as Int32.

The type must contain a single Int64 field.

Specifies to treat particular field of as Int32.

The type must contain field with name and Int64 type.

Specifies to treat all fields of as Int32.

The type must contain only Int64 fields.

Specifies to treat as pointer with appropriate size.

The type must contain a single UInt64 field.

Specifies to treat all fields of as pointers with appropriate size.

The type must contain only UInt64 fields.

Specifies to treat particular field of as pointer with appropriate size.

The type must contain field with name and UInt64 type.

Constructs .

Generic down-ward tree based on chunk array (aka ). Tree format: Tree format: for each node first child and next sibling node are stored. Provides minimal memory footprint for each node and payload can be value type with reference-type fields. But has some performance overhead on accessing nodes.

The non-virtual write-only stream region.

IMPORTANT! Using several regions over the same underlying stream lead to undefined behavior.

A delegate for a -like method, but with the right side of the comparison assumed to be already known by the method.

Accepts right-side value as argument to prevent memory traffic on closures.

Provides the facility to control number of memory-heavy tasks run in parallel.

The helper class for fluent-style actual memory usage declaration.

The helper class for fluent-style expected memory usage declaration.

Awaits while current memory consumption allows to run task with expected memory amount.
IMPORTANT! The single task is allowed to run even requested memory more than physically available.

Declares actual memory usage bound to .

Declares expected memory usage which should be reserved for current task.

Waits for dependent task. On enter decrements number of running tasks to allow sub-task run even memory consumption exceed limits. On exit increments number of running tasks back.

USE this method when necessary to wait sub-task without releasing current memory reservation.

Raised once when some task requested more than physically available memory amount.
Might be used to show user warning.
BE AWARE: the event raised on task's thread.

Numeric reference to tree/graph node.

Parallel ForEach: makes number of forks according to recommended parallelism degree; each fork process next unprocessed item from array by one.

Ths implementation suppose that length of isn't large but each item processing is complex and requires many steps.

Parallel ForEach: makes number of forks according to recommended parallelism degree; each fork process slice of source array.

Ths implementation suppose that length of is large but each item processing is quite simple.

Represents key for item in the persistent storage.

Temporary overrides random seed used for pivot element selection. Use this method if reproducible sort order for equal sequences is needed.

In-place quick sort with random pivot and without recursion.

Sort is in-place, use it carefully if concurrent access to is possible.

In-place quick sort with random pivot and without recursion.

Sort is in-place, use it carefully if concurrent access to is possible.

In-place parallel quick sort with random pivot and without recursion.