#include <Tensor.hpp>

Inheritance diagram for kp::Tensor:

Public Types
enum class	TensorTypes { eDevice = 0 , eHost = 1 , eStorage = 2 }

enum class	TensorDataTypes { eBool = 0 , eInt = 1 , eUnsignedInt = 2 , eFloat = 3 , eDouble = 4 }

Public Member Functions
	Tensor (std::shared_ptr< vk::PhysicalDevice > physicalDevice, std::shared_ptr< vk::Device > device, void *data, uint32_t elementTotalCount, uint32_t elementMemorySize, const TensorDataTypes &dataType, const TensorTypes &tensorType=TensorTypes::eDevice)

virtual	~Tensor ()

void	rebuild (void *data, uint32_t elementTotalCount, uint32_t elementMemorySize)

void	destroy ()

bool	isInit ()

TensorTypes	tensorType ()

void	recordCopyFrom (const vk::CommandBuffer &commandBuffer, std::shared_ptr< Tensor > copyFromTensor)

void	recordCopyFromStagingToDevice (const vk::CommandBuffer &commandBuffer)

void	recordCopyFromDeviceToStaging (const vk::CommandBuffer &commandBuffer)

void	recordPrimaryBufferMemoryBarrier (const vk::CommandBuffer &commandBuffer, vk::AccessFlagBits srcAccessMask, vk::AccessFlagBits dstAccessMask, vk::PipelineStageFlagBits srcStageMask, vk::PipelineStageFlagBits dstStageMask)

void	recordStagingBufferMemoryBarrier (const vk::CommandBuffer &commandBuffer, vk::AccessFlagBits srcAccessMask, vk::AccessFlagBits dstAccessMask, vk::PipelineStageFlagBits srcStageMask, vk::PipelineStageFlagBits dstStageMask)

vk::DescriptorBufferInfo	constructDescriptorBufferInfo ()

uint32_t	size ()

uint32_t	dataTypeMemorySize ()

uint32_t	memorySize ()

TensorDataTypes	dataType ()

void *	rawData ()

void	setRawData (const void *data)

template<typename T >
T *	data ()

template<typename T >
std::vector< T >	vector ()

Static Public Member Functions
static std::string	toString (TensorDataTypes dt)

static std::string	toString (TensorTypes dt)

Protected Attributes
TensorTypes	mTensorType

TensorDataTypes	mDataType

uint32_t	mSize

uint32_t	mDataTypeMemorySize

void *	mRawData

Detailed Description

Structured data used in GPU operations.

Tensors are the base building block in Kompute to perform operations across GPUs. Each tensor would have a respective Vulkan memory and buffer, which would be used to store their respective data. The tensors can be used for GPU data storage or transfer.

Member Enumeration Documentation

◆ TensorTypes

enum kp::Tensor::TensorTypes

strong

Type for tensors created: Device allows memory to be transferred from staging buffers. Staging are host memory visible. Storage are device visible but are not set up to transfer or receive data (only for shader storage).

Enumerator
eDevice	Type is device memory, source and destination.
eHost	Type is host memory, source and destination.
eStorage	Type is Device memory (only)

Constructor & Destructor Documentation

◆ Tensor()

kp::Tensor::Tensor	(	std::shared_ptr< vk::PhysicalDevice >	physicalDevice,
		std::shared_ptr< vk::Device >	device,
		void *	data,
		uint32_t	elementTotalCount,
		uint32_t	elementMemorySize,
		const TensorDataTypes &	dataType,
		const TensorTypes &	tensorType = `TensorTypes::eDevice`
	)

Constructor with data provided which would be used to create the respective vulkan buffer and memory.

Parameters

physicalDevice	The physical device to use to fetch properties
device	The device to use to create the buffer and memory from
data	Non-zero-sized vector of data that will be used by the tensor
tensorTypes	Type for the tensor which is of type TensorTypes

◆ ~Tensor()

virtual kp::Tensor::~Tensor ( )

virtual

Destructor which is in charge of freeing vulkan resources unless they have been provided externally.

Member Function Documentation

◆ constructDescriptorBufferInfo()

vk::DescriptorBufferInfo kp::Tensor::constructDescriptorBufferInfo ( )

Constructs a vulkan descriptor buffer info which can be used to specify and reference the underlying buffer component of the tensor without exposing it.

Returns: Descriptor buffer info with own buffer

◆ data()

template<typename T >

T* kp::Tensor::data ( )

inline

Template to return the pointer data converted by specific type, which would be any of the supported types including float, double, int32, uint32 and bool.

Returns: Pointer to raw memory containing raw bytes data of Tensor.

◆ dataType()

TensorDataTypes kp::Tensor::dataType ( )

Retrieve the data type of the tensor (host, device, storage)

Returns: Data type of tensor of type kp::Tensor::TensorDataTypes

◆ dataTypeMemorySize()

uint32_t kp::Tensor::dataTypeMemorySize ( )

Returns the total size of a single element of the respective data type that this tensor holds.

Returns: Unsigned integer representing the memory of a single element of the respective data type.

◆ destroy()

void kp::Tensor::destroy ( )

Destroys and frees the GPU resources which include the buffer and memory.

◆ isInit()

bool kp::Tensor::isInit ( )

Check whether tensor is initialized based on the created gpu resources.

Returns: Boolean stating whether tensor is initialized

◆ memorySize()

uint32_t kp::Tensor::memorySize ( )

Returns the total memory size of the data contained by the Tensor object which would equate to (this->size() * this->dataTypeMemorySize())

Returns: Unsigned integer representing the memory of a single element of the respective data type.

◆ rawData()

void* kp::Tensor::rawData ( )

Retrieve the raw data via the pointer to the memory that contains the raw memory of this current tensor. This tensor gets changed to a nullptr when the Tensor is removed.

Returns: Pointer to raw memory containing raw bytes data of Tensor.

◆ rebuild()

void kp::Tensor::rebuild	(	void *	data,
		uint32_t	elementTotalCount,
		uint32_t	elementMemorySize
	)

Function to trigger reinitialisation of the tensor buffer and memory with new data as well as new potential device type.

Parameters

data	Vector of data to use to initialise vector from
tensorType	The type to use for the tensor

◆ recordCopyFrom()

void kp::Tensor::recordCopyFrom	(	const vk::CommandBuffer &	commandBuffer,
		std::shared_ptr< Tensor >	copyFromTensor
	)

Records a copy from the memory of the tensor provided to the current thensor. This is intended to pass memory into a processing, to perform a staging buffer transfer, or to gather output (between others).

Parameters

commandBuffer	Vulkan Command Buffer to record the commands into
copyFromTensor	Tensor to copy the data from

◆ recordCopyFromDeviceToStaging()

void kp::Tensor::recordCopyFromDeviceToStaging ( const vk::CommandBuffer & commandBuffer )

Records a copy from the internal device memory to the staging memory using an optional barrier to wait for the operation. This function would only be relevant for kp::Tensors of type eDevice.

Parameters

commandBuffer Vulkan Command Buffer to record the commands into

◆ recordCopyFromStagingToDevice()

void kp::Tensor::recordCopyFromStagingToDevice ( const vk::CommandBuffer & commandBuffer )

Records a copy from the internal staging memory to the device memory using an optional barrier to wait for the operation. This function would only be relevant for kp::Tensors of type eDevice.

Parameters

commandBuffer Vulkan Command Buffer to record the commands into

◆ recordPrimaryBufferMemoryBarrier()

void kp::Tensor::recordPrimaryBufferMemoryBarrier	(	const vk::CommandBuffer &	commandBuffer,
		vk::AccessFlagBits	srcAccessMask,
		vk::AccessFlagBits	dstAccessMask,
		vk::PipelineStageFlagBits	srcStageMask,
		vk::PipelineStageFlagBits	dstStageMask
	)

Records the buffer memory barrier into the primary buffer and command buffer which ensures that relevant data transfers are carried out correctly.

Parameters

commandBuffer	Vulkan Command Buffer to record the commands into
srcAccessMask	Access flags for source access mask
dstAccessMask	Access flags for destination access mask
scrStageMask	Pipeline stage flags for source stage mask
dstStageMask	Pipeline stage flags for destination stage mask

◆ recordStagingBufferMemoryBarrier()

void kp::Tensor::recordStagingBufferMemoryBarrier	(	const vk::CommandBuffer &	commandBuffer,
		vk::AccessFlagBits	srcAccessMask,
		vk::AccessFlagBits	dstAccessMask,
		vk::PipelineStageFlagBits	srcStageMask,
		vk::PipelineStageFlagBits	dstStageMask
	)

Records the buffer memory barrier into the staging buffer and command buffer which ensures that relevant data transfers are carried out correctly.

Parameters

commandBuffer	Vulkan Command Buffer to record the commands into
srcAccessMask	Access flags for source access mask
dstAccessMask	Access flags for destination access mask
scrStageMask	Pipeline stage flags for source stage mask
dstStageMask	Pipeline stage flags for destination stage mask

◆ setRawData()

void kp::Tensor::setRawData ( const void * data )

Sets / resets the data of the tensor which is directly done on the GPU host visible memory available by the tensor.

◆ size()

uint32_t kp::Tensor::size ( )

Returns the size/magnitude of the Tensor, which will be the total number of elements across all dimensions

Returns: Unsigned integer representing the total number of elements

◆ tensorType()

TensorTypes kp::Tensor::tensorType ( )

Retrieve the tensor type of the Tensor

Returns: Tensor type of tensor

◆ vector()

template<typename T >

std::vector<T> kp::Tensor::vector ( )

inline

Template to get the data of the current tensor as a vector of specific type, which would be any of the supported types including float, double, int32, uint32 and bool.

Returns: Vector of type provided by template.

The documentation for this class was generated from the following file:

/home/alejandro/Programming/kubernetes/seldon/vulkan-kompute/src/include/kompute/Tensor.hpp

Public Types

Public Member Functions

Static Public Member Functions

Protected Attributes

Detailed Description

Member Enumeration Documentation

◆ TensorTypes

Constructor & Destructor Documentation

◆ Tensor()

◆ ~Tensor()

Member Function Documentation

◆ constructDescriptorBufferInfo()

◆ data()

◆ dataType()

◆ dataTypeMemorySize()

◆ destroy()

◆ isInit()

◆ memorySize()

◆ rawData()

◆ rebuild()

◆ recordCopyFrom()

◆ recordCopyFromDeviceToStaging()

◆ recordCopyFromStagingToDevice()

◆ recordPrimaryBufferMemoryBarrier()

◆ recordStagingBufferMemoryBarrier()

◆ setRawData()

◆ size()

◆ tensorType()

◆ vector()