• UE4 LLM工具


    LLM(Low Level Memory Tracker) 是从 4.18 开始引入的新的内存统计工具,比memreport 统计数据更加详细精确,但又不会像 MallocProfiler 那样有很大的本身开销

    通过LLM,可以知道游戏消耗的内存分别来自引擎的哪些模块,让开发者可以对内存占用高的模块进行有针对性优化。

    Debug、Development包才会启用ENABLE_LOW_LEVEL_MEM_TRACKER宏,统计逻辑才会编译进可执行二进制中,才能使用LLM相关的功能

    Shipping包是没有LLM功能,对于Test包可以#define ALLOW_LOW_LEVEL_MEM_TRACKER_IN_TEST 1来将LLM代码编译进包

    相关逻辑在UnrealEngineEngineSourceRuntimeCorePublicHALLowLevelMemTracker.hUnrealEngineEngineSourceRuntimeCorePrivateHALLowLevelMemTracker.cpp文件中

    LLM目前有两个Tracker(跟踪器):Default Tracker(默认跟踪器)Platform Tracker(平台跟踪器)。它们都有自己的独立分配映射和标记堆栈。

    Default Tracker(ELLMTracker::Default)的级别较高,记录引擎中FMemoryMalloc函数进行的分配。为 stat LLM和 stat LLMFULL控制台命令提供统计信息。

    Platform Tracker(ELLMTracker::Platform)的级别较低,记录从OS进行的所有分配。例如,它会跟踪FMemoryMalloc函数进行的内部分配。因此,Default Tracker是Platform Tracker统计信息的子集。

    运行开关

    在编译时打开 LLM 后,要想在运行时使用 LLM 的功能,还要加上启动命令行 -LLM -LLMCSV

    -LLM:运行时打开LLM统计    或者通过定义#define LLM_AUTO_ENABLE 1默认打开

    -LLMCSV:将内存统计信息输出到CSV 文件中,CSV 文件保存在 SavedProfilingLLM 目录下  注:设置控制台变量LLM.LLMWriteInterval xx,修改写csv文件的频率为xx秒,默认5秒写一行

    会得到两个 csv 文件,一个是 LLM_ 前缀,另一个是 LLMPlatform_前缀,这两个文件分别由两个 Tracker 生成,这两个文件的关注点不同

    ① LLM 主要关注FMalloc内存的消费者,主要就是引擎内各个部分的内存开销

    ② LLMPlatform 只关注系统内存的消费者,主要有几大类: ProgramSize, FMalloc, LLMOverhead

    -LLMTAGSETS=Assets:显示每个资源分配的统计,不只是有大类别的数据。   注:需打开LLM_ALLOW_ASSETS_TAGS

    -LLMTAGSETS=AssetClasses:按UObject类别进行分类统计

    注:写CSV时,如果开启了-LLMTAGSETS 命令行,CSV 文件可能会错乱。

           因为写 CSV 文件时预留了第一行标题栏内容的缓冲区,当每个资源或资源类很多,会导致标题栏内容溢出,覆盖了后面的数据区域,所以这里需要自己按需求修改一下

           具体的控制台命令为:LLM.LLMHeaderMaxSize 500000

    运行时开关 LLM 的处理见 FLowLevelMemTracker::ProcessCommandLine 函数

    在UE4CommandLine.txt文件中添加命令行参数:

    ../../../MyGame/MyGame.uproject -LLM -LLMCSV

    Android需要将该文件push到手机的UE4Game/MyGame/UE4CommandLine.txt

    iOS需要将ue4commandline.txt注:需要全小写)放在如下目录中:

    从csv生成html图表

    PerfreportTool.exe -csv F:UAGameSavedProfilingLLMLLM_Pid23989_2021.03.30-14.14.55.csv -o c:LLMOutput -reporttype LLM -topng. -graphxml LLMReportGraphs.xml -reportxml LLMReportTypes.xml -nostripevents

    PerfreportTool.exe -csv F:UAGameSavedProfilingLLMLLMPlatform_Pid23989_2021.03.30-14.14.55.csv -o c:LLMOutput -reporttype LLM -topng. -graphxml LLMReportGraphs.xml -reportxml LLMReportTypes.xml -nostripevents

    生成的html文件如下:

    统计项说明

    LLM_Pid23989_2021.03.30-14.14.55内容如下:

    Untagged,RenderingThread,Audio/AudioMisc,EngineInit,Stats,LoadMapMisc,Audio/AudioMixer,GC,Audio,AsyncLoading,UObject,FileSystem,Materials,Shaders,TextureMetaData,InitUObject,UI,EnginePreInit,Textures,Meshes,SceneRender,RenderTargets,RHIMisc,Meshes/StaticMesh,FName,MaterialInstance,PSO,Particles,Localization,ConfigSystem,TaskGraphMiscTasks,Physics/PhysXAllocator,Physics,AssetRegistry,CsvProfiler,Audio/AudioMixerPlugins,Physics/PhysX,NavigationRecast,EngineMisc,ProgramSize,GenericPlatformMallocCrash,FMallocUnused,TrackedTotal,Total,Untracked,StreamingManager,Lua,Meshes/Landscape,Animation,Meshes/SkeletalMesh,Niagara,MediaStreaming,Meshes/InstancedMesh,Physics/PhysXLandscape,Audio/AudioSynthesis, 
    。。。 。。。
    35.32,6.21,0.02,0.90,21.65,5.05,9.52,0.62,9.55,3.08,105.19,13.72,0.54,19.11,0.65,12.84,42.70,18.71,67.28,31.85,34.72,40.85,0.80,8.79,7.12,0.31,0.15,0.78,36.59,2.80,0.00,6.32,6.71,0.13,0.00,0.00,0.39,0.71,3.78,157.18,3.61,55.31,789.20,830.58,41.44,0.44,28.88,0.03,13.38,12.33,0.00,0.00,0.13,0.00,0.00,
    35.32,6.21,0.02,0.90,21.52,5.05,9.52,0.62,9.55,3.08,105.22,13.72,0.54,19.11,0.65,12.84,42.71,18.71,67.28,31.78,34.25,40.85,0.80,8.79,7.12,0.31,0.15,0.78,36.59,2.80,0.00,6.33,6.71,0.13,0.00,0.00,0.39,0.71,3.78,157.18,3.61,56.45,789.74,842.68,52.88,0.44,28.91,0.03,13.38,12.27,0.00,0.00,0.13,0.00,0.00,
    35.32,6.21,0.02,0.90,21.65,5.05,9.52,0.62,9.55,3.08,105.22,13.72,0.54,19.11,0.65,12.84,42.74,18.71,67.28,31.89,34.99,40.85,0.80,8.79,7.12,0.31,0.15,0.78,36.59,2.80,0.00,6.34,6.73,0.13,0.00,0.00,0.39,0.71,3.79,157.18,3.61,55.68,790.05,828.77,38.50,0.44,28.94,0.03,13.38,12.37,0.00,0.00,0.13,0.00,0.00,
    35.32,6.21,0.02,0.90,21.59,5.05,9.52,0.62,9.55,3.08,105.25,13.72,0.54,19.11,0.65,12.84,42.72,18.71,67.28,31.86,35.13,40.85,0.80,8.79,7.12,0.31,0.15,0.78,36.59,2.80,0.00,6.32,6.71,0.13,0.00,0.00,0.39,0.71,3.81,157.18,3.61,55.56,790.05,841.99,52.15,0.44,28.96,0.03,13.44,12.34,0.00,0.00,0.13,0.00,0.00,
    35.32,6.21,0.02,0.90,21.40,5.05,9.52,0.65,9.55,3.08,105.18,13.72,0.54,19.11,0.65,12.84,42.72,18.71,67.28,31.32,35.28,40.85,0.80,8.79,7.12,0.31,0.15,0.78,36.59,2.80,0.00,6.34,6.73,0.13,0.00,0.00,0.39,0.71,3.77,157.18,3.61,54.50,788.20,838.28,50.08,0.44,28.99,0.03,13.26,11.80,0.00,0.00,0.13,0.00,0.00,
    35.32,6.21,0.02,0.90,21.84,5.05,9.52,0.65,9.55,3.08,105.18,13.72,0.54,19.11,0.65,12.84,42.74,18.71,67.28,31.59,35.64,40.85,0.80,8.79,7.12,0.31,0.15,0.78,36.59,2.80,0.00,6.34,6.73,0.13,0.00,0.00,0.39,0.71,3.77,157.18,3.61,55.04,789.89,837.26,46.91,0.44,29.02,0.03,13.26,12.07,0.00,0.00,0.13,0.00,0.00,
    35.32,6.21,0.02,0.90,22.03,5.05,9.52,0.65,9.55,3.08,105.17,13.72,0.54,19.11,0.65,12.84,42.75,18.71,67.28,31.52,35.90,40.85,0.80,8.79,7.12,0.31,0.15,0.78,36.59,2.80,0.00,6.34,6.73,0.13,0.00,0.00,0.39,0.71,3.74,157.18,3.61,54.95,790.17,831.60,40.49,0.44,29.05,0.03,13.26,11.96,0.00,0.00,0.13,0.00,0.00,
    35.32,6.21,0.02,0.90,21.46,5.05,9.52,0.65,9.55,3.08,105.19,13.72,0.54,19.11,0.65,12.84,42.73,18.71,67.28,31.74,35.36,40.85,0.80,8.79,7.12,0.31,0.15,0.78,36.59,2.80,0.00,6.34,6.73,0.13,0.00,0.00,0.39,0.71,3.75,157.18,3.61,55.90,790.32,843.07,53.02,0.44,29.07,0.03,13.31,12.22,0.00,0.00,0.13,0.00,0.00,
    35.32,6.21,0.02,0.90,21.46,5.05,9.52,0.65,9.55,3.08,105.22,13.72,0.54,19.11,0.65,12.84,42.74,18.71,67.28,31.82,35.44,40.85,0.80,8.79,7.12,0.31,0.15,0.78,36.59,2.80,0.00,6.34,6.73,0.13,0.00,0.00,0.39,0.71,3.77,157.18,3.61,56.08,790.86,843.15,52.41,0.44,29.23,0.03,13.32,12.30,0.00,0.00,0.13,0.00,0.00,
    35.32,6.21,0.02,0.90,21.78,5.05,9.52,0.65,9.55,3.08,105.23,13.72,0.54,19.11,0.65,12.84,42.75,18.71,67.28,31.71,36.19,40.85,0.80,8.79,7.12,0.31,0.15,0.78,36.59,2.80,0.00,6.33,6.72,0.13,0.00,0.00,0.39,0.71,3.77,157.18,3.61,55.50,791.27,842.89,51.62,0.44,29.26,0.03,13.32,12.18,0.00,0.00,0.13,0.00,0.00,

    LLMPlatform_Pid23989_2021.03.30-14.14.55.csv内容如下:

    Untagged,FMalloc,Graphics,ProgramSize,GenericPlatformMallocCrash,TrackedTotal,Total,Untracked,LLMOverhead,OSAvailable,
    。。。 。。。
    0.00,511.05,115.27,157.18,3.61,918.22,961.69,43.47,131.11,3389.45,
    0.00,511.61,115.27,157.18,3.61,907.32,962.20,54.88,119.64,3388.55,
    0.00,511.86,115.28,157.18,3.61,920.53,961.36,40.84,132.58,3387.90,
    0.00,511.86,115.27,157.18,3.61,907.62,961.63,54.02,119.69,3386.14,
    0.00,510.05,115.27,157.18,3.61,905.81,957.96,52.16,119.69,3389.23,
    0.00,511.74,115.27,157.18,3.61,907.49,956.95,49.46,119.69,3387.49,
    0.00,511.99,115.31,157.18,3.61,919.30,962.82,43.52,131.22,3385.75,
    0.00,512.18,115.27,157.18,3.61,907.96,962.80,54.84,119.72,3388.29,
    0.00,512.68,115.27,157.18,3.61,908.46,962.87,54.41,119.72,3386.87,
    0.00,513.11,115.27,157.18,3.61,908.93,962.64,53.72,119.75,3386.62,

    以第1条采集的值为例(单位MB),说明各统计项含义:

    Untagged // 未分类的内存,即分配内存时候不在 LLM_SCOPE 宏范围内
    ELLMTag::Untagged(Default Tracker): 35.32
    ELLMTag::Untagged(Platform Tracker): 0

    LLMOverhead // LLM本身消耗的内存
    (Default Tracker): 无
    ELLMTag::PlatformOverhead(Platform Tracker): 119.64

    TrackedTotal // 通过调用 LLM OnLowLevelAlloc/OnLowLevelFree 统计到的所有内存
    ELLMTag::TrackedTotal(Default Tracker):789.74
    ELLMTag::PlatformTrackedTotal(Platform Tracker):907.32
    注:统计到的内存值ELLMTag::PlatformTrackedTotal比ELLMTag::TrackedTotal要大

    Total // 总内存
    ELLMTag::PlatformTotal(Platform Tracker):962.2
    在 Android 上为 RSS,在 iOS 上为 resident_size,在其它平台上为系统所有进程消耗的内存(不明白这里为什么是这样),所以这个值目前只有在移动端有参考意义
    ELLMTag::Total(Default Tracker):842.68
    PlatformTotal - LLMOverhead。即为上面的内存减去 LLM 本身消耗的内存,这个内存值可以认为是在不打开 LLM 功能下的进程内存值,同上,这个值目前只在移动端上有参考意义

    Untracked // 未知的内存分配,即 LLM 无法追踪的内存,比如第三方库里的内存分配
    ELLMTag::PlatformUntracked(Platform Tracker): 54.88  // PlatformTotal - PlatformTrackedTotal
    ELLMTag::Untracked(Default Tracker):52.88  // Total - TrackedTotal

    FMalloc // 通过FMemory内存分配器分配的内存,也就是一般意义上引擎分配的内存,比如各种UObject,纹理,动画等占用的内存
    (Default Tracker): 无
    ELLMTag::FMalloc(Platform Tracker): 511.61

    FMallocUnused  // ELLMTag::FMalloc内存分配器的内存 - 通过FMemory得到的内存。即内存分配器中未使用的内存
    ELLMTag::FMallocUnused(Default Tracker): 56.45
    (Platform Tracker): 无
    注:LLM追踪的内存有2种ELLMAllocType::FMalloc和ELLMAllocType::System

    ProgramSize // 程序刚起来时占用的内存,即刚进 Main 函数时,在 Android 上为 RSS,在 iOS 上为 resident_size
    ELLMTag::ProgramSize(Default Tracker):157.18
    ELLMTag::ProgramSizePlatform(Platform Tracker):157.18

    Graphics  // 显存
    (Default Tracker): 无
    ELLMTag::GraphicsPlatform(Platform Tracker): 115.27

    GenericPlatformMallocCrash // 预分配的 crash 相关的内存,当 crash 发生时用来替换GMalloc
    ELLMTag::GenericPlatformMallocCrash(Default Tracker):3.61
    ELLMTag::GenericPlatformMallocCrashPlatform(Platform Tracker):3.61

    OSAvailable // 系统当前可用的物理内存
    (Default Tracker): 无
    ELLMTag::PlatformOSAvailable(Platform Tracker): 3388.55

    OOMBackupPool  // Out of memory 备用内存池
    ELLMTag::BackupOOMMemoryPool(Default Tracker):未统计
    ELLMTag::BackupOOMMemoryPoolPlatform(Platform Tracker): 未统计

    Default Tracker详细的统计项
    ELLMTag::EngineMisc 3.78 // 在以下函数执行过程中分配的内存
    AActor::ProcessEvent
    UPackageMapClient::SerializeNewActor
    FEngineLoop::Tick

    ELLMTag::TaskGraphTasksMisc 0 // FTaskThreadAnyThread::ProcessTasks 函数执行过程中分配的内存

    Audio  9.52 // 音频相关的内存,包括音频资源,音频线程中分配的内存,具体还包含以下子类别
    ELLMTag::AudioMisc   0.02
    ELLMTag::AudioSoundWaves
    ELLMTag::AudioMixer   9.52
    ELLMTag::AudioPrecache
    ELLMTag::AudioDecompress
    ELLMTag::AudioRealtimePrecache
    ELLMTag::AudioFullDecompress
    ELLMTag::AudioVoiceChat

    ELLMTag::FName 7.12 // FName占用的内存
    ELLMTag::Networking // 网络相关的内存,包含网络包处理,连接对象内存等

    ELLMTag::Meshes 31.85 // Mesh 的顶点缓存,顶点索引缓存占用的内存,还有以下子类别
    ELLMTag::StaticMesh 8.79 // StaticMesh 相关的内存,包含 StaticMesh 资源,StaticMeshComponent
    ELLMTag::SkeletalMesh 12.33 // SkeletalMeshComponent, SkinnedMeshComponent, SkeletalMesh内存
    ELLMTag::InstancedMesh 0.13 // InstancedStaticMeshComponent, HierarchicalInstancedStaticMeshComponent内存

    ELLMTag::Landscape // LandscapeComponent, LandscapeProxy, LandscapeComponentSceneProxy内存
    ELLMTag::Stats 21.65 // stat 统计相关的内存占用
    ELLMTag::Shaders 19.11 // 各种类型 Shander 的内存占用
    ELLMTag::PSO 0.15 // Pipeline State Object 缓存的内存占用
    ELLMTag::Textures 67.28 // 纹理相关的内存占用
    ELLMTag::TextureMetaData 0.65 // UTexture2D::Serialize 函数执行过程中分配的内存
    ELLMTag::RenderTargets 40.85 // RT 相关的内存,包含在 Viewport 中渲染使用的内存
    ELLMTag::SceneRender 34.72 // 场景渲染相关的内存占用,包含 Slate RHI Render 的窗口渲染
    ELLMTag::RHIMisc 0.8 // 其它渲染相关的内存占用,如各种渲染状态,RHI Thread 运行过程中的内存
    ELLMTag::AsyncLoading 3.08 // 异步资源加载过程中的内存占用,包含 AsyncLoadingThread, Event Driven Loader
    ELLMTag::UObject 105.19 // UObject 占用的内存,包含 UClass 等
    ELLMTag::Animation 13.38 // 动画相关的内存,包含 AnimInstance, AnimSequence, AnimationAsset, AnimBlueprint, MorphTarget
    ELLMTag::Materials 0.54 // 材质相关的内存,包含 MaterialInterface, MaterialFunction
    ELLMTag::Particles 0.78 // 特效相关的内存,包含 ParticleSystemComponent

    ELLMTag::GC 0.62 // GC 过程中的内存,分别为以下两个函数
    CollectGarbageInternal
    PerformReachabilityAnalysis

    ELLMTag::UI 42.7 // Slate 相关的内存,包含字体,TextureAtlas
    ELLMTag::PhysX // PhysX 物理相关的内存
    ELLMTag::EnginePreInitMemory 18.71 // FEngineLoop::PreInit 函数执行过程中的内存
    ELLMTag::EngineInitMemory 0.9 // FEngineLoop::Init 函数执行过程中的内存
    ELLMTag::RenderingThreadMemory 6.21 // 渲染线程(RenderingThreadMain)执行过程中的内存

    ELLMTag::LoadMapMisc 5.05   // 地图加载过程中的内存,分别为以下两个函数
    UEngine::LoadMap
    UWorld::UpdateLevelStreaming

    ELLMTag::StreamingManager 0.44 // StreamableManager 相关的内存,主要是资源 streaming 函数执行过程中的内存

    ELLMTag::FileSystem 13.72 // 文件系统相关的内存,主要是文件读取时的缓冲区,比如Pak 文件
    ELLMTag::Localization 36.59 // 本地化相关的内存
    ELLMTag::AssetRegistry 0.13 // AssetRegistryModule 内存
    ELLMTag::ConfigSystem 2.8 // 配置文件相关的内存
    ELLMTag::InitUObject 12.84 // InitUObject函数执行过程中的内存
    ELLMTag::MaterialInstance 0.31 // MaterialInstance, MaterialInstanceDynamic包括材质参数内存占用
    ELLMTag::Lua 28.88 // Lua 内存

    Windows下特有统计项

    ELLMTag::WorkingSetSize  // 进程物理内存,可用这个值替代上面的PlatformTotal
    ELLMTag::PagefileUsed  // 进程虚拟内存

    ThreadStack  // Windows 线程栈内存
    ELLMTag::ThreadStack(Platform Tracker)
    ELLMTag::ThreadStackPlatform(Default Tracker)

    llm stat控制台命令

    stat llm   // 显示 Default Tracker 分组的简要内存统计信息

    stat llmfull  // 显示 Default Tracker 分组的所有内存统计信息

    注1:默认一个Group最多显示的行数为25,可通过执行控制台变量stats.MaxPerGroup xx来指定新的最大行数为xx

    注2:为了能查看显示不下的内容,自定义开发了stat scroll控制台命令

    stat scroll 1 // 向下滚动1行
    stat scroll -1 // 向上滚动1行
    stat scroll 999 // 滚动到底部
    stat scroll -999 // 滚动到顶部
    stat scroll restore //恢复初始

     Stat LLMPlatform   // 显示 Platform Tracker 分组的内存统计信息

    Stat LLMOverhead  // 显示 LLM 本身使用的内存信息

    参考

    UE4 Low Level Memory Tracker 使用(知乎)

    UE4 Low Level Memory Tracker 使用

  • 相关阅读:
    .net经典笔试题
    asp.net面试题
    Java 面 试 题
    .net面试题集
    J2EE面试题集锦
    linux下查看硬件信息
    linux下软件的安装[转]
    在Linux系统下优化Oracle具体步骤
    制作Linux的优盘启动盘
    两端对齐,图片文字列表
  • 原文地址:https://www.cnblogs.com/kekec/p/14593345.html
Copyright © 2020-2023  润新知