Skip to content

[LLDB][Data Formatters] Calculate average and total time for summary providers within lldb #102708

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 24 commits into from
Sep 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
b451c16
Initial attempt at new classes Summary statistics in SummaryStatistic…
Jlalond Aug 8, 2024
d69df7c
Implement inovcation timing for summary providers
Jlalond Aug 9, 2024
48f373f
Sprinkle new entry into tests where there is nothing in the array, an…
Jlalond Aug 9, 2024
0e0828c
Added more tests for python script provider and added vector as a tes…
Jlalond Aug 14, 2024
16be363
Include cleanup of unused method in typesummary.h
Jlalond Aug 14, 2024
62a1622
Remove the empty line that keeps getting included
Jlalond Aug 14, 2024
19ab63f
Move to std::string vs const string, and have the cache directly give…
Jlalond Aug 14, 2024
8ae8b02
Switch to string map, set the summary string directly for script summ…
Jlalond Aug 14, 2024
c96b8b7
Run formatters
Jlalond Aug 14, 2024
faa5ba7
Talked with Greg and we decided to move back to not having the cache …
Jlalond Aug 15, 2024
7aaf218
Stop always creating a new SP to summary stats per try insert. Remove…
Jlalond Aug 15, 2024
28fb346
Run GCF
Jlalond Aug 15, 2024
a0172fd
Actually rename variables in test after changing json key names
Jlalond Aug 19, 2024
4a2f234
Rebase and implement Michael's feedback on std::move in the summary s…
Jlalond Aug 19, 2024
72c6f2f
Reformat code and add note
Jlalond Aug 19, 2024
7c0894d
Implement more feedback on naming, and finally add some multithreaded…
Jlalond Aug 22, 2024
4ffeef9
Format
Jlalond Aug 22, 2024
dcaeda6
Remove comment
Jlalond Aug 23, 2024
479e6b5
Add comment why SummaryStatSp isn't forward declared
Jlalond Aug 23, 2024
9f9134e
Final tidy up feedback and run clang-format
Jlalond Aug 23, 2024
aa71d5f
Rework tests a bit to support the CI introduced edgecase where someti…
Jlalond Sep 4, 2024
f63080b
Move to a unified function for the kind name
Jlalond Sep 5, 2024
c847083
Run py formatter
Jlalond Sep 9, 2024
3cccbba
Make box summary more generic so it works on all CI hosts
Jlalond Sep 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions lldb/include/lldb/DataFormatters/TypeSummary.h
Original file line number Diff line number Diff line change
Expand Up @@ -258,6 +258,14 @@ class TypeSummaryImpl {

virtual std::string GetDescription() = 0;

/// Get the name of the Type Summary Provider, either a C++ class, a summary
/// string, or a script function name.
virtual std::string GetName() = 0;

/// Get the name of the kind of Summary Provider, either c++, summary string,
/// script or python.
virtual std::string GetSummaryKindName();

uint32_t &GetRevision() { return m_my_revision; }

typedef std::shared_ptr<TypeSummaryImpl> SharedPointer;
Expand Down Expand Up @@ -293,6 +301,8 @@ struct StringSummaryFormat : public TypeSummaryImpl {

std::string GetDescription() override;

std::string GetName() override;

static bool classof(const TypeSummaryImpl *S) {
return S->GetKind() == Kind::eSummaryString;
}
Expand Down Expand Up @@ -340,6 +350,8 @@ struct CXXFunctionSummaryFormat : public TypeSummaryImpl {
return S->GetKind() == Kind::eCallback;
}

std::string GetName() override;

typedef std::shared_ptr<CXXFunctionSummaryFormat> SharedPointer;

private:
Expand All @@ -352,6 +364,7 @@ struct CXXFunctionSummaryFormat : public TypeSummaryImpl {
struct ScriptSummaryFormat : public TypeSummaryImpl {
std::string m_function_name;
std::string m_python_script;
std::string m_script_formatter_name;
StructuredData::ObjectSP m_script_function_sp;

ScriptSummaryFormat(const TypeSummaryImpl::Flags &flags,
Expand Down Expand Up @@ -384,6 +397,8 @@ struct ScriptSummaryFormat : public TypeSummaryImpl {

std::string GetDescription() override;

std::string GetName() override;

static bool classof(const TypeSummaryImpl *S) {
return S->GetKind() == Kind::eScript;
}
Expand Down
80 changes: 80 additions & 0 deletions lldb/include/lldb/Target/Statistics.h
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
#ifndef LLDB_TARGET_STATISTICS_H
#define LLDB_TARGET_STATISTICS_H

#include "lldb/DataFormatters/TypeSummary.h"
#include "lldb/Utility/ConstString.h"
#include "lldb/Utility/RealpathPrefixes.h"
#include "lldb/Utility/Stream.h"
Expand All @@ -17,6 +18,7 @@
#include "llvm/Support/JSON.h"
#include <atomic>
#include <chrono>
#include <mutex>
#include <optional>
#include <ratio>
#include <string>
Expand All @@ -26,6 +28,9 @@ namespace lldb_private {

using StatsClock = std::chrono::high_resolution_clock;
using StatsTimepoint = std::chrono::time_point<StatsClock>;
class SummaryStatistics;
// Declaring here as there is no private forward
typedef std::shared_ptr<SummaryStatistics> SummaryStatisticsSP;

class StatsDuration {
public:
Expand Down Expand Up @@ -175,6 +180,81 @@ struct StatisticsOptions {
std::optional<bool> m_include_transcript;
};

/// A class that represents statistics about a TypeSummaryProviders invocations
/// \note All members of this class need to be accessed in a thread safe manner
class SummaryStatistics {
public:
explicit SummaryStatistics(std::string name, std::string impl_type)
: m_total_time(), m_impl_type(std::move(impl_type)),
m_name(std::move(name)), m_count(0) {}

std::string GetName() const { return m_name; };
double GetTotalTime() const { return m_total_time.get().count(); }

uint64_t GetSummaryCount() const {
return m_count.load(std::memory_order_relaxed);
}

StatsDuration &GetDurationReference() { return m_total_time; };

std::string GetSummaryKindName() const { return m_impl_type; }

llvm::json::Value ToJSON() const;

/// Basic RAII class to increment the summary count when the call is complete.
class SummaryInvocation {
public:
SummaryInvocation(SummaryStatisticsSP summary_stats)
: m_stats(summary_stats),
m_elapsed_time(summary_stats->GetDurationReference()) {}
~SummaryInvocation() { m_stats->OnInvoked(); }

/// Delete the copy constructor and assignment operator to prevent
/// accidental double counting.
/// @{
SummaryInvocation(const SummaryInvocation &) = delete;
SummaryInvocation &operator=(const SummaryInvocation &) = delete;
/// @}

private:
SummaryStatisticsSP m_stats;
ElapsedTime m_elapsed_time;
};

private:
void OnInvoked() noexcept { m_count.fetch_add(1, std::memory_order_relaxed); }
lldb_private::StatsDuration m_total_time;
const std::string m_impl_type;
const std::string m_name;
std::atomic<uint64_t> m_count;
};

/// A class that wraps a std::map of SummaryStatistics objects behind a mutex.
class SummaryStatisticsCache {
public:
/// Get the SummaryStatistics object for a given provider name, or insert
/// if statistics for that provider is not in the map.
SummaryStatisticsSP
GetSummaryStatisticsForProvider(lldb_private::TypeSummaryImpl &provider) {
std::lock_guard<std::mutex> guard(m_map_mutex);
if (auto iterator = m_summary_stats_map.find(provider.GetName());
iterator != m_summary_stats_map.end())
return iterator->second;

auto it = m_summary_stats_map.try_emplace(
provider.GetName(),
std::make_shared<SummaryStatistics>(provider.GetName(),
provider.GetSummaryKindName()));
return it.first->second;
}

llvm::json::Value ToJSON();

private:
llvm::StringMap<SummaryStatisticsSP> m_summary_stats_map;
std::mutex m_map_mutex;
};

/// A class that represents statistics for a since lldb_private::Target.
class TargetStats {
public:
Expand Down
5 changes: 5 additions & 0 deletions lldb/include/lldb/Target/Target.h
Original file line number Diff line number Diff line change
Expand Up @@ -1224,6 +1224,10 @@ class Target : public std::enable_shared_from_this<Target>,

void ClearAllLoadedSections();

lldb_private::SummaryStatisticsSP GetSummaryStatisticsSPForProviderName(
lldb_private::TypeSummaryImpl &summary_provider);
lldb_private::SummaryStatisticsCache &GetSummaryStatisticsCache();

/// Set the \a Trace object containing processor trace information of this
/// target.
///
Expand Down Expand Up @@ -1557,6 +1561,7 @@ class Target : public std::enable_shared_from_this<Target>,
std::string m_label;
ModuleList m_images; ///< The list of images for this process (shared
/// libraries and anything dynamically loaded).
SummaryStatisticsCache m_summary_statistics_cache;
SectionLoadHistory m_section_load_history;
BreakpointList m_breakpoint_list;
BreakpointList m_internal_breakpoint_list;
Expand Down
12 changes: 11 additions & 1 deletion lldb/source/Core/ValueObject.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -615,7 +615,17 @@ bool ValueObject::GetSummaryAsCString(TypeSummaryImpl *summary_ptr,
m_synthetic_value->UpdateValueIfNeeded(); // the summary might depend on
// the synthetic children being
// up-to-date (e.g. ${svar%#})
summary_ptr->FormatObject(this, destination, actual_options);

if (TargetSP target_sp = GetExecutionContextRef().GetTargetSP()) {
SummaryStatisticsSP stats_sp =
target_sp->GetSummaryStatisticsCache()
.GetSummaryStatisticsForProvider(*summary_ptr);

// Construct RAII types to time and collect data on summary creation.
SummaryStatistics::SummaryInvocation invocation(stats_sp);
summary_ptr->FormatObject(this, destination, actual_options);
} else
summary_ptr->FormatObject(this, destination, actual_options);
}
m_flags.m_is_getting_summary = false;
return !destination.empty();
Expand Down
33 changes: 31 additions & 2 deletions lldb/source/DataFormatters/TypeSummary.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,19 @@ TypeSummaryOptions::SetCapping(lldb::TypeSummaryCapping cap) {
TypeSummaryImpl::TypeSummaryImpl(Kind kind, const TypeSummaryImpl::Flags &flags)
: m_flags(flags), m_kind(kind) {}

std::string TypeSummaryImpl::GetSummaryKindName() {
switch (m_kind) {
case Kind::eSummaryString:
return "string";
case Kind::eCallback:
return "callback";
case Kind::eScript:
return "python";
case Kind::eInternal:
return "c++";
}
}

StringSummaryFormat::StringSummaryFormat(const TypeSummaryImpl::Flags &flags,
const char *format_cstr)
: TypeSummaryImpl(Kind::eSummaryString, flags), m_format_str() {
Expand Down Expand Up @@ -116,6 +129,8 @@ std::string StringSummaryFormat::GetDescription() {
return std::string(sstr.GetString());
}

std::string StringSummaryFormat::GetName() { return m_format_str; }

CXXFunctionSummaryFormat::CXXFunctionSummaryFormat(
const TypeSummaryImpl::Flags &flags, Callback impl, const char *description)
: TypeSummaryImpl(Kind::eCallback, flags), m_impl(impl),
Expand Down Expand Up @@ -145,15 +160,27 @@ std::string CXXFunctionSummaryFormat::GetDescription() {
return std::string(sstr.GetString());
}

std::string CXXFunctionSummaryFormat::GetName() { return m_description; }

ScriptSummaryFormat::ScriptSummaryFormat(const TypeSummaryImpl::Flags &flags,
const char *function_name,
const char *python_script)
: TypeSummaryImpl(Kind::eScript, flags), m_function_name(),
m_python_script(), m_script_function_sp() {
if (function_name)
// Take preference in the python script name over the function name.
if (function_name) {
m_function_name.assign(function_name);
if (python_script)
m_script_formatter_name = function_name;
}
if (python_script) {
m_python_script.assign(python_script);
m_script_formatter_name = python_script;
}

// Python scripts include the tabbing of the function def so we remove the
// leading spaces.
m_script_formatter_name = m_script_formatter_name.erase(
0, m_script_formatter_name.find_first_not_of(' '));
}

bool ScriptSummaryFormat::FormatObject(ValueObject *valobj, std::string &retval,
Expand Down Expand Up @@ -201,3 +228,5 @@ std::string ScriptSummaryFormat::GetDescription() {
}
return std::string(sstr.GetString());
}

std::string ScriptSummaryFormat::GetName() { return m_script_formatter_name; }
20 changes: 20 additions & 0 deletions lldb/source/Target/Statistics.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,8 @@ TargetStats::ToJSON(Target &target,
m_source_realpath_attempt_count);
target_metrics_json.try_emplace("sourceRealpathCompatibleCount",
m_source_realpath_compatible_count);
target_metrics_json.try_emplace("summaryProviderStatistics",
target.GetSummaryStatisticsCache().ToJSON());
return target_metrics_json;
}

Expand Down Expand Up @@ -420,3 +422,21 @@ llvm::json::Value DebuggerStats::ReportStatistics(

return std::move(global_stats);
}

llvm::json::Value SummaryStatistics::ToJSON() const {
return json::Object{{
{"name", GetName()},
{"type", GetSummaryKindName()},
{"count", GetSummaryCount()},
{"totalTime", GetTotalTime()},
}};
}

json::Value SummaryStatisticsCache::ToJSON() {
std::lock_guard<std::mutex> guard(m_map_mutex);
json::Array json_summary_stats;
for (const auto &summary_stat : m_summary_stats_map)
json_summary_stats.emplace_back(summary_stat.second->ToJSON());

return json_summary_stats;
}
10 changes: 10 additions & 0 deletions lldb/source/Target/Target.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -3194,6 +3194,16 @@ bool Target::SetSectionUnloaded(const lldb::SectionSP &section_sp,

void Target::ClearAllLoadedSections() { m_section_load_history.Clear(); }

lldb_private::SummaryStatisticsSP Target::GetSummaryStatisticsSPForProviderName(
lldb_private::TypeSummaryImpl &summary_provider) {
return m_summary_statistics_cache.GetSummaryStatisticsForProvider(
summary_provider);
}

SummaryStatisticsCache &Target::GetSummaryStatisticsCache() {
return m_summary_statistics_cache;
}

void Target::SaveScriptedLaunchInfo(lldb_private::ProcessInfo &process_info) {
if (process_info.IsScriptedProcess()) {
// Only copy scripted process launch options.
Expand Down
15 changes: 15 additions & 0 deletions lldb/test/API/commands/statistics/basic/BoxFormatter.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import lldb


def summary(valobj, dict):
return f"[{valobj.GetChildAtIndex(0).GetValue()}]"


def __lldb_init_module(debugger, dict):
typeName = "Box<.*$"
debugger.HandleCommand(
'type summary add -x "'
+ typeName
+ '" --python-function '
+ f"{__name__}.summary"
)
2 changes: 1 addition & 1 deletion lldb/test/API/commands/statistics/basic/Makefile
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
C_SOURCES := main.c
CXX_SOURCES := main.cpp
include Makefile.rules
Loading