gostor / gotgt

gotgt The gotgt project is a simple SCSI Target framework implemented in golang built for performance and density. Very briefly, this iSCSI/SCSI target Go implementation can be included/imported as a library to allow upper layer iSCSI clients to communicate to the actual SCSI devices. The target configuration is static with a json file for the time being. The core functionality of this target library provides the iSCSI/SCSI protocol services. A simple flat file based LUN target implementation is provided with plug-in interface. In the future, a formal plugin mechanism will be provided and supported to work with more sophisticated backend storage arrays. What is SCSI? Small Computer System Interface (SCSI) is a set of standards for physically connecting and transferring data between computers and peripheral devices. The SCSI standards define commands, protocols, electrical and optical interfaces. SCSI is most commonly used for hard disk drives and tape drives, but it can connect a wide range of other devices, including scanners and CD drives, although not all controllers can handle all devices. What is iSCSI? The iSCSI is an acronym for Internet Small Computer Systems Interface, an Internet Protocol (IP)-based storage networking standard for linking data storage facilities. In a nutshell, it provides block-level access to storage devices over a TCP/IP network. Getting started Currently, the gotgt is under heavy development, so there is no any release binaries so far, you have to build it from source. There is a only one binary name , you can start a daemon via and control it via . Build You will need to make sure that you have Go installed on your system and the automake package is installed also. The repository should be cloned in your $GOPATH. How to use Now, there is lack of commands to operate the target and LU, however you can init the target/LU with config file in , you may find a example at here. Please note, if you want use that example, you have to make sure file exists. A quick overview of the source code The source code repository is right now organized into two main portions, i.e., the cmd and the pkg directories. The cmd directory implementation is intended to manage targets, LUNs and TPGTs, which includes create, remove and list actions. It provides these functionalities through a daemon. In the future, when fully enhanced and implemented, it would take RESTful syntax as well. The pkg directory has three main pieces, i.e., the API interface, the SCSI layer and the iSCSI target layer. The API interface provides management services such as create and remove targets. The iSCSI target layer implements the protocol required to receive and transmit iSCSI PDU's, and communicates with the SCSI layer to carry out SCSI commands and processing. The SCSI layer implements the SCSI SPC and SBC standards that talks to the SCSI devices attached to the target library. Note that the examples directory is intended to show static configurations that serve as the backend storage. The simplest configuration has one LUN and one flat file behind the LUN in question. This json configuration file is read once at the beginning of the iSCSI target library instantiation. Performance Optimizations gotgt includes several performance optimizations for high-throughput and low-latency storage workloads: • NUMA-Aware Memory Allocation For multi-socket systems, gotgt can optimize memory allocation to use NUMA-local memory, reducing cross-socket memory access latency. **Features:** • Automatic NUMA topology detection • NUMA-local buffer pools for I/O operations • Thread pinning to specific NUMA nodes • Configurable per-node buffer pool sizing **Configuration:** • io_uring Backend Storage (Linux 5.1+) On Linux systems with kernel 5.1 or later, gotgt can use io_uring for high-performance asynchronous I/O, bypassing the traditional Linux AIO interface. **Features:** • Asynchronous I/O using io_uring • Reduced system call overhead • Better performance for high queue depth workloads • Automatic fallback to standard I/O on older kernels **Requirements:** • Linux kernel 5.1 or later • x86_64, ARM64, or other supported architectures **Configuration:** **Backend Type Options:** • - Standard file I/O (default) • - io_uring-based I/O (Linux 5.1+) • Object Pooling The iSCSI protocol layer uses sync.Pool for efficient object reuse: • ISCSICommand object pooling to reduce GC pressure • Buffer pooling for protocol header processing • NUMA-aware buffer allocation for data operations • Combined High-Performance Configuration Example For maximum performance, combine both NUMA and io_uring: • Performance Tuning Tips • **NUMA Optimization**: On multi-socket systems, ensure the iSCSI target threads run on the same NUMA node as the storage devices • **Queue Depth**: For NVMe or fast SSDs, increase to 4096 or higher • **Buffer Sizes**: Match to your typical I/O size (e.g., 64KB, 128KB, 256KB) • **CPU Pinning**: Use to pin storage backends to specific NUMA nodes • Benchmarking Use fio to benchmark performance: For more details, see PERFORMANCE_OPTIMIZATIONS.md. Test You can test this with open-iscsi or libiscsi. For more information and example test scripts, please refer to the test directory. SCSI Commands Support For a complete list of supported SCSI commands, see SCSI_COMMANDS.md. Roadmap The current roadmap and milestones for alpha and beta completion are in the github issues on this repository. Please refer to these issues for what is being worked on and completed for the various stages of development. Contributing Want to help build gotgt? Check out our contributing documentation.