DMA Accelerator Functional Unit User Guide: Intel® FPGA Programmable Acceleration Card D5005

ID 683270
Date 8/03/2020
Give Feedback

8. Optimization for Improved DMA Performance

Implementation of NUMA (non-uniform memory access) optimization in fpga_dma_test.cpp allows the processor to access its own local memory faster than accessing non-local memory (memory local to another processor).

A typical NUMA configuration is shown in the diagram below. The local access represents access from a core to memory local to the same core. The remote access illustrates the path taken when a core on Node 0 accesses memory that resides in memory local to Node 1.
Figure 6. Typical NUMA Configuration
Use the following code to implement NUMA optimization in your test application:
// Set up proper affinity if requested
	if (cpu_affinity || memory_affinity) {
		unsigned dom = 0, bus = 0, dev = 0, func = 0;
		fpga_properties props;
		int retval;
				char str[4096];
		res = fpgaGetProperties(afc_token, &props);
		ON_ERR_GOTO(res, out_destroy_tok, "fpgaGetProperties");
		res = fpgaPropertiesGetBus(props, (uint8_t *) & bus);
		ON_ERR_GOTO(res, out_destroy_tok, "fpgaPropertiesGetBus");
		res = fpgaPropertiesGetDevice(props, (uint8_t *) & dev);
		ON_ERR_GOTO(res, out_destroy_tok, "fpgaPropertiesGetDevice");
		res = fpgaPropertiesGetFunction(props, (uint8_t *) & func);
		ON_ERR_GOTO(res, out_destroy_tok, "fpgaPropertiesGetFunction");

		// Find the device from the topology
		hwloc_topology_t topology;
		hwloc_obj_t obj = hwloc_get_pcidev_by_busid(topology, dom, bus, dev, func);
		hwloc_obj_t obj2 = hwloc_get_non_io_ancestor_obj(topology, obj);
			hwloc_obj_type_snprintf(str, 4096, obj2, 1);
			printf("%s\n", str);
			hwloc_obj_attr_snprintf(str, 4096, obj2, " :: ", 1);
			printf("%s\n", str);
			hwloc_bitmap_taskset_snprintf(str, 4096, obj2->cpuset);
			printf("CPUSET is %s\n", str);
			hwloc_bitmap_taskset_snprintf(str, 4096, obj2->nodeset);
			printf("NODESET is %s\n", str);
		if (memory_affinity) {
			#if HWLOC_API_VERSION > 0x00020000
				retval = hwloc_set_membind(topology, obj2->nodeset,
				retval =
				hwloc_set_membind_nodeset(topology, obj2->nodeset,
			ON_ERR_GOTO(retval, out_destroy_tok, "hwloc_set_membind");
		if (cpu_affinity) {
			retval = hwloc_set_cpubind(topology, obj2->cpuset,	HWLOC_CPUBIND_STRICT);
			ON_ERR_GOTO(retval, out_destroy_tok, "hwloc_set_cpubind");