Interesting kind of part of the system, such as a Core, a Cache, a Memory node, etc. The different types detected by hwloc are detailed in the hwloc_obj_type_t enumeration.
--
They are topologically sorted by CPU set into a tree.
--
--
--
CPU set
--
The set of logical processors (or processing units) logically included in an object (if it makes sense). They are always expressed using physical logical processor numbers (as announced by the OS). They are implemented as the hwloc_bitmap_t opaque structure. hwloc CPU sets are just masks, they do not have any relation with an operating system actual binding notion like Linux' cpusets.
--
--
--
Node set
--
The set of NUMA memory nodes logically included in an object (if it makes sense). They are always expressed using physical node numbers (as announced by the OS). They are implemented with the hwloc_bitmap_t opaque structure. as bitmaps.
--
--
--
Bitmap
--
A possibly-infinite set of bits used for describing sets of objects such as CPUs (CPU sets) or memory nodes (Node sets). They are implemented with the hwloc_bitmap_t opaque structure.
--
--
--
Parent object
--
The object logically containing the current object, for example because its CPU set includes the CPU set of the current object.
--
--
--
Ancestor object
--
The parent object, or its own parent object, and so on.
--
--
--
Children object(s)
--
The object (or objects) contained in the current object because their CPU set is included in the CPU set of the current object.
--
--
--
Arity
--
The number of children of an object.
--
--
--
Sibling objects
--
Objects of the same type which have the same parent.
--
--
--
Sibling rank
--
Index to uniquely identify objects of the same type which have the same parent, and is always in the range [0, parent_arity).
--
--
--
Cousin objects
--
Objects of the same type as the current object.
--
--
--
Level
--
Set of objects of the same type.
--
--
--
OS or physical index
--
The index that the operating system (OS) uses to identify the object. This may be completely arbitrary, or it may depend on the BIOS configuration.
--
--
--
Depth
--
Nesting level in the object tree, starting from the 0th object.
--
--
--
Logical index
--
Index to uniquely identify objects of the same type. It expresses proximity in a generic way. This index is always linear and in the range [0, num_objs_same_type_same_level). Think of it as ``cousin rank.'' The ordering is based on topology first, and then on OS CPU numbers, so it is stable across everything except firmware CPU renumbering.
--
--
--
Logical processor
--
--
Processing unit
--
The smallest processing element that can be represented by a hwloc object. It may be a single-core processor, a core of a multicore processor, or a single thread in SMT processor.
--
--
--
--
The following diagram can help to understand the vocabulary of the relationships by showing the example of a machine with two dual core sockets (with no hardware threads); thus, a topology with 4 levels. Each box with rounded corner corresponds to one hwloc_obj_t, containing the values of the different integer fields (depth, logical_index, etc.), and arrows show to which other hwloc_obj_t pointers point to (first_child, parent, etc.)
--
--
--
--
It should be noted that for PU objects, the logical index -- as computed linearly by hwloc -- is not the same as the OS index.
hwloc comes with an extensive C programming interface and several command line utilities. Each of them is fully documented in its own manual page; the following is a summary of the available command line tools.
--
--lstopo
--
lstopo (also known as hwloc-info and hwloc-ls) displays the hierarchical topology map of the current system. The output may be graphical or textual, and can also be exported to numerous file formats such as PDF, PNG, XML, and others.
--
Note that lstopo can read XML files and/or alternate chroot filesystems and display topological maps representing those systems (e.g., use lstopo to output an XML file on one system, and then use lstopo to read in that XML file and display it on a different system).
--
--hwloc-bind
--
hwloc-bind binds processes to specific hardware objects through a flexible syntax. A simple example is binding an executable to specific cores (or sockets or bitmaps or ...). The hwloc-bind(1) man page provides much more detail on what is possible.
--
hwloc-bind can also be used to retrieve the current process' binding.
--
--hwloc-calc
--
hwloc-calc is generally used to create bitmap strings to pass to hwloc-bind. Although hwloc-bind accepts many forms of object specification (i.e., bitmap strings are one of many forms that hwloc-bind understands), they can be useful, compact representations in shell scripts, for example.
--
hwloc-calc generates bitmap strings from given hardware objects with the ability to aggregate them, intersect them, and more. hwloc-calc generally uses the same syntax than hwloc-bind, but multiple instances may be composed to generate complex combinations.
--
Note that hwloc-calc can also generate lists of logical processors or NUMA nodes that are convenient to pass to some external tools such as taskset or numactl.
--
--hwloc-distrib
--
hwloc-distrib generates a set of bitmap strings that are uniformly distributed across the machine for the given number of processes. These strings may be used with hwloc-bind to run processes to maximize their memory bandwidth by properly distributing them across the machine.
--
--hwloc-ps
--
hwloc-ps is a tool to display the bindings of processes that are currently running on the local machine. By default, hwloc-ps only lists processes that are bound; unbound process (and Linux kernel threads) are not displayed.
The behavior of the hwloc library and tools may be tuned thanks to the following environment variables.
--
--
HWLOC_XMLFILE=/path/to/file.xml
--
enforces the discovery from the given XML file as if hwloc_topology_set_xml() had been called. This file may have been generated earlier with lstopo file.xml. For convenience, this backend provides empty binding hooks which just return success. To have hwloc still actually call OS-specific hooks, HWLOC_THISSYSTEM should be set 1 in the environment too, to assert that the loaded file is really the underlying system.
--
--
--
HWLOC_FSROOT=/path/to/linux/filesystem-root/
--
switches to reading the topology from the specified Linux filesystem root instead of the main file-system root, as if hwloc_topology_set_fsroot() had been called. Not using the main file-system root causes hwloc_topology_is_thissystem() to return 0. For convenience, this backend provides empty binding hooks which just return success. To have hwloc still actually call OS-specific hooks, HWLOC_THISSYSTEM should be set 1 in the environment too, to assert that the loaded file is really the underlying system.
--
--
--
HWLOC_THISSYSTEM=1
--
enforces the return value of hwloc_topology_is_thissystem(). It means that it makes hwloc assume that the selected backend provides the topology for the system on which we are running, even if it is not the OS-specific backend but the XML backend for instance. This means making the binding functions actually call the OS-specific system calls and really do binding, while the XML backend would otherwise provide empty hooks just returning success. This can be used for efficiency reasons to first detect the topology once, save it to an XML file, and quickly reload it later through the XML backend, but still having binding functions actually do bind.
--
--
--
--
--Generated on Thu Dec 16 2010 16:48:29 for Hardware Locality (hwloc) by
--
-- 1.7.1
--
--
-diff -uNr hwloc-1.1/doc/doxygen-doc/html/a00004.html hwloc-1.1rc6r2967_final/doc/doxygen-doc/html/a00004.html
---- hwloc-1.1/doc/doxygen-doc/html/a00004.html 2010-12-16 22:48:29.000000000 +0100
-+++ hwloc-1.1rc6r2967_final/doc/doxygen-doc/html/a00004.html 1970-01-01 01:00:00.000000000 +0100
-@@ -1,36 +0,0 @@
--
--
--
--
--Hardware Locality (hwloc): CPU Binding and Memory Binding
--
--
--
--
--
--
Some OSes do not systematically provide separate functions for CPU and Memory binding. This means that CPU binding functions may have have effects on the memory binding policy, and changing the memory binding policy may change the CPU binding of the current thread. This is often not a problem for the application, so by default hwloc will make use of these functions when they provide better binding support.
--
If the application does not want any CPU binding change when changing the memory policy, it needs to use the HWLOC_MEMBIND_NOCPUBIND flag to prevent hwloc from using OS functions which would change the CPU binding. Conversely, HWLOC_CPUBIND_NOMEMBIND can be passed to cpu binding function to prevent hwloc form using OS functions woudl change the memory binding policy. Of course, this will thus reduce hwloc's support for binding, so their use is discouraged.
--
One can however avoid using these flags but still closely control both memory and CPU binding, by allocating memory and touching it, and then changing the CPU binding. The already-really-allocated memory will not be migrated, thus even if the memory binding policy gets changed by the CPU binding order, the effect will have been achieved. On binding and allocating further memory, the CPU binding should be performed again in case the memory binding altered the previously-selected CPU binding.
--
Not all OSes support the notion of a current memory binding policy for the current process but those often still provide a way to allocate data on a given node set. Conversely, some OSes support the notion of a current memory binding policy, and do not permit to allocate data on a given node set without just changing the current policy and allocate the data. Hwloc provides functions that set the current memory binding policies (if supported) as well as functions which allocat memory bound to given node set. By default, it does not use the former to achieve the latter, so that users can use both on OSes where they are both supported, and get both effects at the same time. For convenience, hwloc however also provides the hwloc_alloc_membind_policy and hwloc_alloc_membind_policy_nodeset helpers which are allowed to change the current memory binding policy of the process, in order to achieve memory binding even if that means having to change the current memory binding policy.
--
--Generated on Thu Dec 16 2010 16:48:29 for Hardware Locality (hwloc) by
--
-- 1.7.1
--
--
-diff -uNr hwloc-1.1/doc/doxygen-doc/html/a00005.html hwloc-1.1rc6r2967_final/doc/doxygen-doc/html/a00005.html
---- hwloc-1.1/doc/doxygen-doc/html/a00005.html 2010-12-16 22:48:29.000000000 +0100
-+++ hwloc-1.1rc6r2967_final/doc/doxygen-doc/html/a00005.html 1970-01-01 01:00:00.000000000 +0100
-@@ -1,64 +0,0 @@
--
--
--
--
--Hardware Locality (hwloc): Interoperability with other software
--
--
--
--
--
--
Although hwloc offers its own portable interface, it still may have to interoperate with specific or non-portable libraries that manipulate similar kinds of objects. hwloc therefore offers several specific "helpers" to assist converting between those specific interfaces and hwloc.
--
Some external libraries may be specific to a particular OS; others may not always be available. The hwloc core therefore generally does not explicitly depend on these types of libraries. However, when a custom application uses or otherwise depends on such a library, it may optionally include the corresponding hwloc helper to extend the hwloc interface with dedicated helpers.
--
--
Linux specific features
--
hwloc/linux.h offers Linux-specific helpers that utilize some non-portable features of the Linux system, such as binding threads through their thread ID ("tid") or parsing kernel CPU mask files.
--
--
--
Linux libnuma
--
hwloc/linux-libnuma.h provides conversion helpers between hwloc CPU sets and libnuma-specific types, such as nodemasks and bitmasks. It helps you use libnuma memory-binding functions with hwloc CPU sets.
--
--
--
Glibc
--
hwloc/glibc-sched.h offers conversion routines between Glibc and hwloc CPU sets in order to use hwloc with functions such as sched_setaffinity().
--
--
--
OpenFabrics Verbs
--
hwloc/openfabrics-verbs.h helps interoperability with the OpenFabrics Verbs interface. For example, it can return a list of processors near an OpenFabrics device.
--
--
--
Myrinet Express
--
hwloc/myriexpress.h offers interoperability with the Myrinet Express interface. It can return the list of processors near a Myrinet board managed by the MX driver.
--
--
--
NVIDIA CUDA
--
hwloc/cuda.h and hwloc/cudart.h enable interoperability with NVIDIA CUDA Driver and Runtime interfaces. For instance, it may return the list of processors near NVIDIA GPUs.
--
--
--
Taskset command-line tool
--
The taskset command-line tool is widely used for binding processes. It manipulates CPU set strings in a format that is slightly different from hwloc's one (it does not divide the string in fixed-size subsets and separates them with commas). To ease interoperability, hwloc offers routines to convert hwloc CPU sets from/to taskset-specific string format. Most hwloc command-line tools also support the --taskset option to manipulate taskset-specific strings.
Like most libraries that mainly fill data structures, hwloc is not thread safe but rather reentrant: all state is held in a hwloc_topology_t instance without mutex protection. That means, for example, that two threads can safely operate on and modify two different hwloc_topology_t instances, but they should not simultaneously invoke functions that modify the same instance. Similarly, one thread should not modify a hwloc_topology_t instance while another thread is reading or traversing it. However, two threads can safely read or traverse the same hwloc_topology_t instance concurrently.
--
When running in multiprocessor environments, be aware that proper thread synchronization and/or memory coherency protection is needed to pass hwloc data (such as hwloc_topology_t pointers) from one processor to another (e.g., a mutex, semaphore, or a memory barrier). Note that this is not a hwloc-specific requirement, but it is worth mentioning.
--
For reference, hwloc_topology_t modification operations include (but may not be limited to):
Also references to objects inside the topology are not valid anymore after these functions return.
--
--
--
Runtime topology modifications
--
hwloc_topology_insert_misc_object_by_* (see Tinker with topologies.) may modify the topology significantly by adding objects inside the tree, changing the topology depth, etc.
--
Although references to former objects may still be valid after insertion, it is strongly advised to not rely on any such guarantee and always re-consult the topology to reacquire new instances of objects.
--
--
--
Locating topologies
--
hwloc_topology_ignore*, hwloc_topology_set* (see Configure Topology Detection) do not modify the topology directly, but they do modify internal structures describing the behavior of the next invocation of hwloc_topology_load(). Hence, all of these functions should not be used concurrently.
--
Note that these functions do not modify the current topology until it is actually reloaded; it is possible to use them while other threads are only read the current topology.
--
--
--
--
--Generated on Thu Dec 16 2010 16:48:29 for Hardware Locality (hwloc) by
--
-- 1.7.1
--
--
-diff -uNr hwloc-1.1/doc/doxygen-doc/html/a00007.html hwloc-1.1rc6r2967_final/doc/doxygen-doc/html/a00007.html
---- hwloc-1.1/doc/doxygen-doc/html/a00007.html 2010-12-16 22:48:29.000000000 +0100
-+++ hwloc-1.1rc6r2967_final/doc/doxygen-doc/html/a00007.html 1970-01-01 01:00:00.000000000 +0100
-@@ -1,95 +0,0 @@
--
--
--
--
--Hardware Locality (hwloc): Embedding hwloc in other software
--
--
--
--
--
--
It can be desirable to include hwloc in a larger software package (be sure to check out the LICENSE file) so that users don't have to separately download and install it before installing your software. This can be advantageous to ensure that your software uses a known-tested/good version of hwloc, or for use on systems that do not have hwloc pre-installed.
--
When used in "embedded" mode, hwloc will:
--
--
not install any header files
--
not build any documentation files
--
not build or install any executables or tests
--
not build libhwloc.* -- instead, it will build libhwloc_embedded.*
--
--
There are two ways to put hwloc into "embedded" mode. The first is directly from the configure command line:
The second requires that your software project uses the GNU Autoconf / Automake / Libtool tool chain to build your software. If you do this, you can directly integrate hwloc's m4 configure macro into your configure script. You can then invoke hwloc's configuration tests and build setup by calling an m4 macro (see below).
--
--Using hwloc's M4 Embedding Capabilities
--
Every project is different, and there are many different ways of integrating hwloc into yours. What follows is one example of how to do it.
--
If your project uses recent versions Autoconf, Automake, and Libtool to build, you can use hwloc's embedded m4 capabilities. We have tested the embedded m4 with projects that use Autoconf 2.65, Automake 1.11.1, and Libtool 2.2.6b. Slightly earlier versions of may also work but are untested. Autoconf versions prior to 2.65 are almost certain to not work.
--
You can either copy all the config/hwloc*m4 files from the hwloc source tree to the directory where your project's m4 files reside, or you can tell aclocal to find more m4 files in the embedded hwloc's "config" subdirectory (e.g., add "-Ipath/to/embedded/hwloc/config" to your Makefile.am's ACLOCAL_AMFLAGS).
--
The following macros can then be used from your configure script (only HWLOC_SETUP_CORE must be invoked if using the m4 macros):
--
--
HWLOC_SETUP_CORE(config-dir-prefix, action-upon-success, action-upon-failure, print_banner_or_not): Invoke the hwloc configuration tests and setup the hwloc tree to build. The first argument is the prefix to use for AC_OUTPUT files -- it's where the hwloc tree is located relative to $top_srcdir. Hence, if your embedded hwloc is located in the source tree at contrib/hwloc, you should pass [contrib/hwloc] as the first argument. If HWLOC_SETUP_CORE and the rest of configure completes successfully, then "make" traversals of the hwloc tree with standard Automake targets (all, clean, install, etc.) should behave as expected. For example, it is safe to list the hwloc directory in the SUBDIRS of a higher-level Makefile.am. The last argument, if not empty, will cause the macro to display an announcement banner that it is starting the hwloc core configuration tests.
--
--
HWLOC_SETUP_CORE will set the following environment variables and AC_SUBST them: HWLOC_EMBEDDED_CFLAGS, HWLOC_EMBEDDED_CPPFLAGS, and HWLOC_EMBEDDED_LIBS. These flags are filled with the values discovered in the hwloc-specific m4 tests, and can be used in your build process as relevant. The _CFLAGS, _CPPFLAGS, and _LIBS variables are necessary to build libhwloc (or libhwloc_embedded) itself.
--
HWLOC_SETUP_CORE also sets HWLOC_EMBEDDED_LDADD environment variable (and AC_SUBSTs it) to contain the location of the libhwloc_embedded.la convenience Libtool archive. It can be used in your build process to link an application or other library against the embedded hwloc library.
--
NOTE: If the HWLOC_SET_SYMBOL_PREFIX macro is used, it must be invoked before HWLOC_SETUP_CORE.
--
--
HWLOC_BUILD_STANDALONE: HWLOC_SETUP_CORE defaults to building hwloc in an "embedded" mode (described above). If HWLOC_BUILD_STANDALONE is invoked *before* HWLOC_SETUP_CORE, the embedded definitions will not apply (e.g., libhwloc.la will be built, not libhwloc_embedded.la).
--
--
--
HWLOC_SET_SYMBOL_PREFIX(foo_): Tells the hwloc to prefix all of hwloc's types and public symbols with "foo_"; meaning that function hwloc_init() becomes foo_hwloc_init(). Enum values are prefixed with an upper-case translation if the prefix supplied; HWLOC_OBJ_SYSTEM becomes FOO_HWLOC_OBJ_SYSTEM. This is recommended behavior if you are including hwloc in middleware -- it is possible that your software will be combined with other software that links to another copy of hwloc. If both uses of hwloc utilize different symbol prefixes, there will be no type/symbol clashes, and everything will compile, link, and run successfully. If you both embed hwloc without changing the symbol prefix and also link against an external hwloc, you may get multiple symbol definitions when linking your final library or application.
--
--
--
HWLOC_SETUP_DOCS, HWLOC_SETUP_UTILS, HWLOC_SETUP_TESTS: These three macros only apply when hwloc is built in "standalone" mode (i.e., they should NOT be invoked unless HWLOC_BUILD_STANDALONE has already been invoked).
--
--
--
HWLOC_DO_AM_CONDITIONALS: If you embed hwloc in a larger project and build it conditionally with Automake (e.g., if HWLOC_SETUP_CORE is invoked conditionally), you must unconditionally invoke HWLOC_DO_AM_CONDITIONALS to avoid warnings from Automake (for the cases where hwloc is not selected to be built). This macro is necessary because hwloc uses some AM_CONDITIONALs to build itself, and AM_CONDITIONALs cannot be defined conditionally. Note that it is safe (but unnecessary) to call HWLOC_DO_AM_CONDITIONALS even if HWLOC_SETUP_CORE is invoked unconditionally. If you are not using Automake to build hwloc, this macro is unncessary (and will actually cause errors because it invoked AM_* macros that will be undefined).
--
--
NOTE: When using the HWLOC_SETUP_CORE m4 macro, it may be necessary to explicitly invoke AC_CANONICAL_TARGET (which requires config.sub and config.guess) and/or AC_USE_SYSTEM_EXTENSIONS macros early in the configure script (e.g., after AC_INIT but before AM_INIT_AUTOMAKE). See the Autoconf documentation for further information.
--
Also note that hwloc's top-level configure.ac script uses exactly the macros described above to build hwloc in a standalone mode (by default). You may want to examine it for one example of how these macros are used.
--
--Example Embedding hwloc
--
Here's an example of integrating with a larger project named sandbox that already uses Autoconf, Automake, and Libtool to build itself:
--
--# First, cd into the sandbox project source tree
--shell$ cd sandbox
--shell$ cp -r /somewhere/else/hwloc-<version> my-embedded-hwloc
--shell$ edit Makefile.am
-- 1. Add "-Imy-embedded-hwloc/config" to ACLOCAL_AMFLAGS
-- 2. Add "my-embedded-hwloc" to SUBDIRS
-- 3. Add "$(HWLOC_EMBEDDED_LDADD)" and "$(HWLOC_EMBEDDED_LIBS)" to
-- sandbox's executable's LDADD line. The former is the name of the
-- Libtool convenience library that hwloc will generate. The latter
-- is any dependent support libraries that may be needed by
-- $(HWLOC_EMBEDDED_LDADD).
-- 4. Add "$(HWLOC_EMBEDDED_CFLAGS)" to AM_CFLAGS
-- 5. Add "$(HWLOC_EMBEDDED_CPPFLAGS)" to AM_CPPFLAGS
--shell$ edit configure.ac
-- 1. Add "HWLOC_SET_SYMBOL_PREFIX(sandbox_hwloc_)" line
-- 2. Add "HWLOC_SETUP_CORE([my-embedded-hwloc], [happy=yes], [happy=no])" line
-- 3. Add error checking for happy=no case
--shell$ edit sandbox.c
-- 1. Add #include <hwloc.h>
-- 2. Add calls to sandbox_hwloc_init() and other hwloc API functions
--
Now you can bootstrap, configure, build, and run the sandbox as normal -- all calls to "sandbox_hwloc_*" will use the embedded hwloc rather than any system-provided copy of hwloc.
--
--Generated on Thu Dec 16 2010 16:48:29 for Hardware Locality (hwloc) by
--
-- 1.7.1
--
--
-diff -uNr hwloc-1.1/doc/doxygen-doc/html/a00008.html hwloc-1.1rc6r2967_final/doc/doxygen-doc/html/a00008.html
---- hwloc-1.1/doc/doxygen-doc/html/a00008.html 2010-12-16 22:48:29.000000000 +0100
-+++ hwloc-1.1rc6r2967_final/doc/doxygen-doc/html/a00008.html 1970-01-01 01:00:00.000000000 +0100
-@@ -1,55 +0,0 @@
--
--
--
--
--Hardware Locality (hwloc): Switching from PLPA to hwloc
--
--
--
--
--
--
Although PLPA and hwloc share some of the same ideas, their programming interfaces are quite different. After much debate, it was decided not to emulate the PLPA API with hwloc's API because hwloc's API is already far more rich than PLPA's.
--
More specifically, exploiting modern computing architecture requires the flexible functionality provided by the hwloc API -- the PLPA API is too rigid in its definitions and practices to handle the evolving server hardware landscape (e.g., PLPA only understands cores and sockets; hwloc understands a much larger set of hardware objects).
--
As such, even though it is fully possible to emulate the PLPA API with hwloc (e.g., only deal with sockets and cores), and while the documentation below describes how to do this, we encourage any existing PLPA application authors to actually re-think their application in terms of more than just sockets and cores. In short, we encourage you to use the full hwloc API to exploit all the hardware.
--
--Topology Context vs. Caching
--
First, all hwloc functions take a topology parameter. This parameter serves as an internal storage for the result of the topology discovery. It replaces PLPA's caching abilities and even lets you manipulate multiple topologies as the same time, if needed.
PLPA was designed to understand only cores and sockets. hwloc offers many more different types of objects (e.g., cores, sockets, hardware threads, NUMA nodes, and others) and stores them within a tree of resources.
hwloc manipulates logical indexes, meaning indexes specified with regard to the ordering of objects in the hwloc-provided hierarchical tree. Physical or OS indexes may be entirely hidden if not strictly required. The reason for this is that physical/OS indexes may change with the OS or with the BIOS version. They may be non-consecutive, multiple objects may have the same physical/OS indexes, making their manipulation tricky and highly non-portable.
--
Note that hwloc tries very hard to always present a hierarchical tree with the same logical ordering, regardless of physical or OS index ordering.
--
It is still possible to retrieve physical/OS indexes through the os_index field of objects, but such practice should be avoided as much as possible for the reasons described above (except perhaps for prettyprinting / debugging purposes).
--
HWLOC_OBJ_PU objects are supposed to have different physical/OS indexes since the OS uses them for binding. The os_index field of these objects provides the identifier that may be used for such binding, and hwloc_get_proc_obj_by_os_index() finds the object associated with a specific OS index.
--
But as mentioned above, we discourage the use of these conversion methods for actual binding. Instead, hwloc offers its own binding model using the cpuset field of objects. These cpusets may be duplicated, modified, combined, etc. (see hwloc/bitmap.h for details) and then passed to hwloc_set_cpubind() for binding.
--
--Counting Specification
--
PLPA offers a countspec parameter to specify whether counting all CPUs, only the online ones or only the offline ones. However, some operating systems do not expose the topology of offline CPUs (i.e., offline CPUs are not reported at all by the OS). Also, some processors may not be visible to the current application due to administrative restrictions. Finally, some processors let you shutdown a single hardware thread in a core, making some of the PLPA features irrelevant.
--
hwloc stores in the hierarchical tree of objects all CPUs that have known topology information. It then provides the applications with several cpusets that contain the list of CPUs that are actually known, that have topology information, that are online, or that are available to the application. These cpusets may be retrieved with hwloc_topology_get_online_cpuset() and other similar functions to filter the object that are relevant or not.
--I do not want hwloc to rediscover my enormous machine topology everytime I rerun a process
--
Although the topology discovery is not expensive on common machines, its overhead may become significant when multiple processes repeat the discovery on large machines (for instance when starting one process per core in a parallel application). The machine topology usually does not vary much, except if some cores are stopped/restarted or if the administrator restrictions are modified. Thus rediscovering the whole topology again and again may look useless.
--
For this purpose, hwloc offers XML import/export features. It lets you save the discovered topology to a file (for instance with the lstopo program) and reload it later by setting the HWLOC_XMLFILE environment variable. Loading a XML topology is usually much faster than querying multiple files or calling multiple functions of the operating system. It is also possible to manipulate such XML files with the C programming interface, and the import/export may also be directed to memory buffer (that may for instance be transmitted between applications through a socket).
This includes the CPUs contained in this object which are allowed for binding, i.e. passing them to the hwloc binding functions should not return permission errors. This is usually restricted by administration rules. Some of them may however be offline so binding to them may still not be possible, see online_cpuset.
--
Note:
Its value must not be changed, hwloc_bitmap_dup must be used instead.
--
--
--
--
--
--
--
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+The CPU set of allowed logical processors.
-+
-+This includes the CPUs contained in this object which are allowed for binding, i.e. passing them to the hwloc binding functions should not return permission errors. This is usually restricted by administration rules. Some of them may however be offline so binding to them may still not be possible, see online_cpuset.
-+
Note:
Its value must not be changed, hwloc_bitmap_dup must be used instead.
This includes the NUMA memory nodes contained in this object which are allowed for memory allocation, i.e. passing them to NUMA node-directed memory allocation should not return permission errors. This is usually restricted by administration rules.
--
If there are no NUMA nodes in the machine, all the memory is close to this object, so allowed_nodeset is full.
--
Note:
Its value must not be changed, hwloc_bitmap_dup must be used instead.
--
--
--
--
--
--
--
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+The set of allowed NUMA memory nodes.
-+
-+This includes the NUMA memory nodes contained in this object which are allowed for memory allocation, i.e. passing them to NUMA node-directed memory allocation should not return permission errors. This is usually restricted by administration rules.
-+If there are no NUMA nodes in the machine, all the memory is close to this object, so allowed_nodeset is full.
-+
Note:
Its value must not be changed, hwloc_bitmap_dup must be used instead.
The complete CPU set of logical processors of this object,.
--
This includes not only the same as the cpuset field, but also the CPUs for which topology information is unknown or incomplete, and the CPUs that are ignored when the HWLOC_TOPOLOGY_FLAG_WHOLE_SYSTEM flag is not set. Thus no corresponding PU object may be found in the topology, because the precise position is undefined. It is however known that it would be somewhere under this object.
--
Note:
Its value must not be changed, hwloc_bitmap_dup must be used instead.
--
--
--
--
--
--
--
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+The complete CPU set of logical processors of this object,.
-+
-+This includes not only the same as the cpuset field, but also the CPUs for which topology information is unknown or incomplete, and the CPUs that are ignored when the HWLOC_TOPOLOGY_FLAG_WHOLE_SYSTEM flag is not set. Thus no corresponding PU object may be found in the topology, because the precise position is undefined. It is however known that it would be somewhere under this object.
-+
Note:
Its value must not be changed, hwloc_bitmap_dup must be used instead.
This includes not only the same as the nodeset field, but also the NUMA nodes for which topology information is unknown or incomplete, and the nodes that are ignored when the HWLOC_TOPOLOGY_FLAG_WHOLE_SYSTEM flag is not set. Thus no corresponding NODE object may be found in the topology, because the precise position is undefined. It is however known that it would be somewhere under this object.
--
If there are no NUMA nodes in the machine, all the memory is close to this object, so complete_nodeset is full.
--
Note:
Its value must not be changed, hwloc_bitmap_dup must be used instead.
--
--
--
--
--
--
--
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+The complete NUMA node set of this object,.
-+
-+This includes not only the same as the nodeset field, but also the NUMA nodes for which topology information is unknown or incomplete, and the nodes that are ignored when the HWLOC_TOPOLOGY_FLAG_WHOLE_SYSTEM flag is not set. Thus no corresponding NODE object may be found in the topology, because the precise position is undefined. It is however known that it would be somewhere under this object.
-+If there are no NUMA nodes in the machine, all the memory is close to this object, so complete_nodeset is full.
-+
Note:
Its value must not be changed, hwloc_bitmap_dup must be used instead.
This is the set of CPUs for which there are PU objects in the topology under this object, i.e. which are known to be physically contained in this object and known how (the children path between this object and the PU objects).
--
If the HWLOC_TOPOLOGY_FLAG_WHOLE_SYSTEM configuration flag is set, some of these CPUs may be offline, or not allowed for binding, see online_cpuset and allowed_cpuset.
--
Note:
Its value must not be changed, hwloc_bitmap_dup must be used instead.
--
--
--
--
--
--
--
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+CPUs covered by this object.
-+
-+This is the set of CPUs for which there are PU objects in the topology under this object, i.e. which are known to be physically contained in this object and known how (the children path between this object and the PU objects).
-+If the HWLOC_TOPOLOGY_FLAG_WHOLE_SYSTEM configuration flag is set, some of these CPUs may be offline, or not allowed for binding, see online_cpuset and allowed_cpuset.
-+
Note:
Its value must not be changed, hwloc_bitmap_dup must be used instead.
NUMA nodes covered by this object or containing this object.
--
This is the set of NUMA nodes for which there are NODE objects in the topology under or above this object, i.e. which are known to be physically contained in this object or containing it and known how (the children path between this object and the NODE objects).
--
In the end, these nodes are those that are close to the current object.
--
If the HWLOC_TOPOLOGY_FLAG_WHOLE_SYSTEM configuration flag is set, some of these nodes may not be allowed for allocation, see allowed_nodeset.
--
If there are no NUMA nodes in the machine, all the memory is close to this object, so nodeset is full.
--
Note:
Its value must not be changed, hwloc_bitmap_dup must be used instead.
--
--
--
--
--
--
--
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+NUMA nodes covered by this object or containing this object.
-+
-+This is the set of NUMA nodes for which there are NODE objects in the topology under or above this object, i.e. which are known to be physically contained in this object or containing it and known how (the children path between this object and the NODE objects).
-+In the end, these nodes are those that are close to the current object.
-+If the HWLOC_TOPOLOGY_FLAG_WHOLE_SYSTEM configuration flag is set, some of these nodes may not be allowed for allocation, see allowed_nodeset.
-+If there are no NUMA nodes in the machine, all the memory is close to this object, so nodeset is full.
-+
Note:
Its value must not be changed, hwloc_bitmap_dup must be used instead.
This includes the CPUs contained in this object that are online, i.e. draw power and can execute threads. It may however not be allowed to bind to them due to administration rules, see allowed_cpuset.
--
Note:
Its value must not be changed, hwloc_bitmap_dup must be used instead.
--
--
--
--
--
--
--
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+
-+The CPU set of online logical processors.
-+
-+This includes the CPUs contained in this object that are online, i.e. draw power and can execute threads. It may however not be allowed to bind to them due to administration rules, see allowed_cpuset.
-+
Note:
Its value must not be changed, hwloc_bitmap_dup must be used instead.
Set of flags describing actual support for this topology.
--
This is retrieved with hwloc_topology_get_support() and will be valid until the topology object is destroyed. Note: the values are correct only after discovery.
--
Field Documentation
--
--
--
--
-+
Detailed Description
-+Flags describing actual discovery support for this topology.
-+
-+Set of flags describing actual support for this topology.
-+
-+This is retrieved with hwloc_topology_get_support() and will be valid until the topology object is destroyed. Note: the values are correct only after discovery.
-+
Convert glibc sched affinity CPU set schedset into hwloc CPU set.
-+
-+
Detailed Description
-+Macros to help interaction between hwloc and glibc scheduling routines.
-+
-+Applications that use both hwloc and glibc scheduling routines such as sched_getaffinity may want to include this file so as to ease conversion between their respective types.
--