MoreMaps.Asyncmap Type

Asyncmap(; ntasks = 100)

Maps concurrently over elements of an array using Base.asyncmap: up to ntasks cooperative tasks in a single process and thread.

Best for:

• IO-bound work (file loading, network requests) where tasks spend most of their time waiting; Threaded wastes cores on such workloads

• Any f that yields (via IO or sleep); CPU-bound f gains nothing here

Usage

julia> using MoreMaps

julia> C = Chart(Asyncmap())
Chart{MoreMaps.All, Asyncmap, NoProgress, NoExpansion}(Asyncmap(100), NoProgress(), NoExpansion())

julia> data = [1, 2, 3, 4, 5];

julia> result = map(x -> x^2, C, data)
5-element Vector{Int64}:
  1
  4
  9
 16
 25

julia> nested_data = [[1, 2], [3, 4], [5, 6]]; # Works with nested arrays

julia> C_nested = Chart(Vector{Int}, Asyncmap(; ntasks = 10));

julia> result = map(sum, C_nested, nested_data)
3-element Vector{Int64}:
  3
  7
 11

Note: Concurrency without parallelism; tasks interleave on one thread whenever f yields. If f never yields, execution is effectively sequential.

See also: Sequential, Threaded, Chart

source

MoreMaps.CallbackLogger Type

CallbackLogger(callback)

A progress logger that calls callback once per completed element. The callback receives a single NamedTuple with fields:

• i: the element index

• done: number of elements completed so far

• total: total number of elements

• y: the value produced for element i

• elapsed: seconds since the map started

The callback always runs on the driver process (on the logger's consumer task), so it can safely mutate driver-local state even under distributed backends.

Usage

julia> using MoreMaps

julia> count = Ref(0);

julia> C = Chart(CallbackLogger(info -> count[] += 1));

julia> map(x -> x^2, C, [1, 2, 3])
3-element Vector{Int64}:
 1
 4
 9

julia> count[]
3

Notes for distributed backends (Pmap, Daggermap): the callback is never shipped to workers (serialization replaces it with a placeholder), so any callback works, including closures over driver-local state. The produced value y is shipped back over the progress channel, so results are serialized twice; if y is large, compute a summary inside f or use QualityLogger-style worker-side scoring instead.

See also: MoreMaps.LogLogger, MoreMaps.CompositeLogger, MoreMaps.Chart

source

MoreMaps.Chart Type

source

MoreMaps.CompositeLogger Type

CompositeLogger(loggers...)

A progress logger that forwards every logging event to each of its child loggers, so several outputs can track the same map (e.g. a terminal bar plus a callback).

Usage

julia> using MoreMaps

julia> counts = Ref(0); values = Float64[];

julia> P = CompositeLogger(
           CallbackLogger(info -> counts[] += 1),
           CallbackLogger(info -> push!(values, info.y))
       );

julia> map(x -> x / 2, Chart(P), [1.0, 2.0, 3.0])
3-element Vector{Float64}:
 0.5
 1.0
 1.5

julia> counts[], sort(values)
(3, [0.5, 1.0, 1.5])

See also: MoreMaps.CallbackLogger, MoreMaps.LogLogger, MoreMaps.Chart

source

MoreMaps.LogLogger Type

LogLogger(; nlogs::Int = 10, level::LogLevel=Info)
LogLogger(nlogs::Int = 10, level::LogLevel=Info)

A progress logger that displays progress information using @info messages. Shows periodic updates during mapping operations.

Arguments

• nlogs::Int: Number of progress messages to display (default: 10)

Usage

julia> using MoreMaps

julia> C = Chart(LogLogger(3))

julia> data = [1, 2, 3, 4, 5, 6];

julia> result = map(x -> (sleep(0.5); x^2), C, data); # Will show progress messages during execution

julia> result

julia> using Logging # Choose a log level

julia> C = Chart(LogLogger(4, Warn));

julia> map(x -> (sleep(0.5); x + 1), C, [1, 2, 3, 4]);

source

MoreMaps.Monitor Type

Monitor()

A progress component that cheaply records resource usage for each map: two clock and GC snapshots per job, nothing per element. Occupies the Chart's progress slot; combine with a real logger via CompositeLogger.

After a map, the fields hold the last job's stats:

• n: number of elements mapped

• time: wall-clock seconds

• bytes: bytes allocated (GC-tracked)

• allocs: number of allocations

• gctime: seconds spent in garbage collection

• backend: the backend instance that ran the job

Usage

julia> using MoreMaps

julia> M = Monitor();

julia> map(x -> x^2, Chart(M), [1, 2, 3]);

julia> M.n
3

julia> M.time >= 0
true

Note: For distributed backends (Pmap, Daggermap), bytes, allocs, and gctime cover the driver process only; worker allocations are not visible. time and n are always faithful.

See also: CompositeLogger, MoreMaps.Chart

source

MoreMaps.NoProgress Type

NoProgress()

The default progress logger that performs no logging.

source

MoreMaps.Pmap Type

Pmap()

Maps concurrently over elements across multiple Julia processes using Distributed.pmap.

Best for:

• Very large arrays

• Memory-intensive operations

• Multi-machine clusters

Usage

julia> using Distributed; addprocs(2);

julia> @everywhere using MoreMaps

julia> C = Chart(Pmap())
Chart{MoreMaps.All, Pmap, NoProgress, NoExpansion}(Pmap(), NoProgress(), NoExpansion())

julia> data = [1, 2, 3, 4, 5];

julia> result = map(x -> x^2, C, data)
5-element Vector{Int64}:
  1
  4
  9
 16
 25

julia> result = map(sum, Chart(Vector{Int}, Pmap()), [[1, 2], [3, 4], [5, 6]])
3-element Vector{Int64}:
  3
  7
 11

Note: Use addprocs() to add worker processes, and @everywhere to load MoreMaps and any functions to be mapped. Functions and data are serialized across processes, which adds overhead.

See also: Sequential, Threaded, Chart

source

MoreMaps.QualityLogger Type

QualityLogger(; nlogs = 0, width = 0, status_width = 20, quality = _default_quality, io = stdout)

Terminal logger that prints rows of colored blocks.

quality(y) may return either:

• Bool (true -> 1.0, false -> 0.0)

• A real-valued score, interpreted in [0, 1] (values are clamped)

Block color bands:

• 0.0: black

• (0.0, 0.25): red

• [0.25, 0.5): orange

• [0.5, 0.75): yellow

• [0.75, 1.0): green

• 1.0: blue

If width == 0, row width defaults to max(floor(Int, sqrt(total)), 50) at runtime. The first status_width characters of each row are reserved for row number + ETA. Set nlogs = 0 to flush every update, or a positive value to flush at that granularity.

source

MoreMaps.Sequential Type

Sequential()

A backend for sequential (single-threaded) execution. Maps one-at-a-time over elements of an array, in order Sequential is the default Chart backend.

Best for:

• Small arrays

• Operations with minimal computational cost

• Debugging and development

Usage

julia> using MoreMaps

julia> C = Chart(Sequential())
Chart{MoreMaps.All, Sequential, NoProgress, NoExpansion}(Sequential(), NoProgress(), NoExpansion())

julia> C = Chart() # Defaults to `Sequential`
Chart{MoreMaps.All, Sequential, NoProgress, NoExpansion}(Sequential(), NoProgress(), NoExpansion())

julia> data = [1, 2, 3, 4, 5];

julia> result = map(x -> x^2, C, data)
5-element Vector{Int64}:
  1
  4
  9
 16
 25


julia> nested_data = [[1, 2], [3, 4], [5, 6]]; # Works with nested arrays

julia> C_nested = Chart(Vector{Int}, Sequential());

julia> result = map(sum, C_nested, nested_data)
3-element Vector{Int64}:
  3
  7
 11

See also: Threaded, Chart

source

MoreMaps.Threaded Type

Threaded()

Maps concurrently over elements of an array using Threads.@threads.

Best for:

• Medium to large arrays

• Single-machine parallelism

Usage

julia> using MoreMaps

julia> C = Chart(Threaded())
Chart{MoreMaps.All, Threaded, NoProgress, NoExpansion}(Threaded(), NoProgress(), NoExpansion())

julia> data = [1, 2, 3, 4, 5];

julia> result = map(x -> x^2, C, data)
5-element Vector{Int64}:
  1
  4
  9
 16
 25

julia> nested_data = [[1, 2], [3, 4], [5, 6]]; # Works with nested arrays

julia> C_nested = Chart(Vector{Int}, Threaded());

julia> result = map(sum, C_nested, nested_data)
3-element Vector{Int64}:
  3
  7
 11

Note: Results may not be in deterministic order due to parallel execution. Use Sequential() if order matters or for debugging. Performance depends on the number of threads available. Start Julia with julia -t auto or set the JULIA_NUM_THREADS environment variable.

See also: Sequential, Chart

source

MoreMaps._run_map Method

_run_map(kernel!, f, C, itrs)

Shared scaffolding for _map implementations. Preallocates the output, initializes the logger, builds the per-element closure g (which calls f and emits a progress log), then invokes kernel!(g, ys, idxs, xs) where ys = nviews(out, idxs) is the writeable view of output leaves. Backends only need to provide kernel!, which drives g over eachindex(idxs) and writes results into ys. Logger lifecycle and exception safety are handled here.

source

MoreMaps.nsimilar Method

Construct a similar nested array to x with new leaves of type outleaf, for original leaves of type inleaf

source

MoreMaps.Daggermap Type

Daggermap(; batchsize = 0, kwargs...)

Maps concurrently over elements of an array using Dagger.jl's task-based parallelism. Daggermap creates a distributed computation graph that can execute across multiple processes and threads.

Elements are grouped into batches of batchsize and one Dagger task is spawned per batch, amortizing the per-task scheduler overhead. batchsize = 0 (the default) picks cld(N, 4 * nprocs()), giving each process about four batches for load balancing. The remaining kwargs are passed to Dagger.Options and apply to each batch task (e.g. scope, single, occupancy).

Best for:

• Very large computations

• Heterogeneous computing resources

• Complex dependency graphs

• Dynamic load balancing

Usage

julia> using MoreMaps, Dagger

julia> C = Chart(Daggermap())
Chart{MoreMaps.All, Daggermap{@NamedTuple{}}, NoProgress, NoExpansion}(Daggermap{@NamedTuple{}}(NamedTuple(), 0), NoProgress(), NoExpansion())

julia> data = [1, 2, 3, 4, 5];

julia> result = map(x -> x^2, C, data)
5-element Vector{Int64}:
  1
  4
  9
 16
 25

julia> nested_data = [[1, 2], [3, 4], [5, 6]]; # Works with nested arrays

julia> C_nested = Chart(Vector{Int}, Daggermap());

julia> result = map(sum, C_nested, nested_data)
3-element Vector{Int64}:
  3
  7
 11

julia> C_opts = Chart(Daggermap(; single = 1, batchsize = 2)); # Options for Dagger tasks

julia> result = map(x -> x + 10, C_opts, [1, 2, 3])
3-element Vector{Int64}:
 11
 12
 13

Note: Uses Dagger.jl's task scheduling, which provides dynamic load balancing and can work across multiple processes. Keyword options are forwarded as Dagger.Options to each spawned batch task.

See also: MoreMaps.Sequential, MoreMaps.Threaded, MoreMaps.Pmap, MoreMaps.Chart

source

MoreMaps.TermLogger Method

TermLogger(nlogs::Int = 0; kwargs...)

A progress logger that creates rich terminal progress bars using Term.jl.

Arguments

• nlogs::Int: Number of update intervals for progress rendering (default: 0, which updates every iteration)

• kwargs...: Additional keyword arguments passed to Term.ProgressBar

Usage

julia> using MoreMaps, Term

julia> P = TermLogger(5);

julia> C = Chart(P);

julia> data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];

julia> result = map(x -> (sleep(0.5); x^2), C, data); # Will display a progress bar

julia> result
10-element Vector{Int64}:
   1
   4
   9
  16
  25
  36
  49
  64
  81
 100

Note: Requires Term.jl to be loaded. Creates visual progress bars in the terminal with customizable appearance. Set nlogs = 0 for maximum update frequency, or higher values to reduce rendering overhead. The progress bar will be transient by default (disappears when complete). Once constructed, a re-used TermLogger will accumulate progress bars from subsequent maps.

See also: MoreMaps.LogLogger, MoreMaps.ProgressLogger, MoreMaps.NoProgress, MoreMaps.Chart

source

MoreMaps.ProgressLogger Method

ProgressLogger(nlogs::Int = 10; id = UUIDs.uuid4(), kwargs...)

A progress logger that integrates with the ProgressLogging.jl ecosystem. Combines LogLogger functionality with ProgressLogging.jl's structured progress reporting. Useful for applications that need standardized progress reporting (e.g., Pluto.jl notebooks, IDEs).

Arguments

• nlogs::Int: Number of progress update intervals (default: 10)

• id: Unique identifier for the progress logger (default: auto-generated UUID)

• kwargs...: Additional keyword arguments passed to ProgressLogging.Progress

Usage

julia> using MoreMaps, ProgressLogging

julia> P = ProgressLogger(5)
ProgressLogger(LogLogger(5), ProgressLogging.Progress(UUIDs.UUID("00000000-0000-0000-0000-000000000000"), "Progress", 1.0, false, :normal, 0, 1.0, Dict{String, Any}(), Any[]))

julia> C = Chart(P)
Chart{MoreMaps.All, Sequential, ProgressLogger, NoExpansion}(Sequential(), ProgressLogger(LogLogger(5), ProgressLogging.Progress(UUIDs.UUID("00000000-0000-0000-0000-000000000000"), "Progress", 1.0, false, :normal, 0, 1.0, Dict{String, Any}(), Any[])), NoExpansion())

julia> data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];

julia> result = map(x -> x^2, C, data); # Will emit ProgressLogging.jl messages

julia> result
10-element Vector{Int64}:
   1
   4
   9
  16
  25
  36
  49
  64
  81
 100

julia> # Works with any backend
       C_threaded = Chart(Threaded(), ProgressLogger(3));

Best with:

• VSCode's progress indicator

• Jupyter notebooks

• Pluto.jl notebooks

• TerminalLoggers.jl

Note: Requires ProgressLogging.jl to be loaded. Progress messages are emitted as structured logs that can be captured by compatible logging systems. Use LogLogger for simple console output or NoProgress to disable progress reporting entirely.

See also: MoreMaps.LogLogger, MoreMaps.NoProgress, MoreMaps.Chart

source