I need to read [100]byte
to transfer a bunch of string
data.
Because not all of the string
s are precisely 100 characters long, the remaining part of the byte array
is padded with 0
s.
If I convert [100]byte
to string
by: string(byteArray[:])
, the tailing 0
s are displayed as ^@^@
s.
In C, the string
will terminate upon 0
, so what's the best way to convert this byte array
to string
in Go?
^@
doesn't show, but it would've been there if you'd test it in the terminal or something similar. The reason for this, is that Go does not stop converting the bytes array to a string when it finds a 0. len(string(bytes))
in your example is 5 and not 1. It depends on the output function, whether the string is fully (with zeros) printed or not.
string(body)
.
Methods that read data into byte slices return the number of bytes read. You should save that number and then use it to create your string. If n
is the number of bytes read, your code would look like this:
s := string(byteArray[:n])
To convert the full string, this can be used:
s := string(byteArray[:len(byteArray)])
This is equivalent to:
s := string(byteArray[:])
If for some reason you don't know n
, you could use the bytes
package to find it, assuming your input doesn't have a null character embedded in it.
n := bytes.Index(byteArray[:], []byte{0})
Or as icza pointed out, you can use the code below:
n := bytes.IndexByte(byteArray[:], 0)
Use:
s := string(byteArray[:])
string(byteArray[:])
contains ^@
characters
string(byteArray)
? Why you need to copy the array using [:]
?
[:]
, for byte arrays, you do.
Simplistic solution:
str := fmt.Sprintf("%s", byteArray)
I'm not sure how performant this is though.
str
has length 100. See play.golang.org/p/XWrmqCbIwkB
For example,
package main
import "fmt"
func CToGoString(c []byte) string {
n := -1
for i, b := range c {
if b == 0 {
break
}
n = i
}
return string(c[:n+1])
}
func main() {
c := [100]byte{'a', 'b', 'c'}
fmt.Println("C: ", len(c), c[:4])
g := CToGoString(c[:])
fmt.Println("Go:", len(g), g)
}
Output:
C: 100 [97 98 99 0]
Go: 3 abc
The following code is looking for '\0', and under the assumptions of the question the array can be considered sorted since all non-'\0' precede all '\0'. This assumption won't hold if the array can contain '\0' within the data.
Find the location of the first zero-byte using a binary search, then slice.
You can find the zero-byte like this:
package main
import "fmt"
func FirstZero(b []byte) int {
min, max := 0, len(b)
for {
if min + 1 == max { return max }
mid := (min + max) / 2
if b[mid] == '\000' {
max = mid
} else {
min = mid
}
}
return len(b)
}
func main() {
b := []byte{1, 2, 3, 0, 0, 0}
fmt.Println(FirstZero(b))
}
It may be faster just to naively scan the byte array looking for the zero-byte, especially if most of your strings are short.
[]byte{0}
. In this case FirstZero()
should return 0
so when slicing result would be ""
, but instead it returns 1
and slicing results in "\x00"
.
When you do not know the exact length of non-nil bytes in the array, you can trim it first:
string(bytes.Trim(arr, "\x00"))
bytes.Trim
takes a slice, not an array (you'd need arr[:]
if arr is actually a [100]byte
as the question states). b) bytes.Trim
is the wrong function to use here. For input like []byte{0,0,'a','b','c',0,'d',0}
it will return "abc\x00d" instead of "" c) there already is a correct answer that uses bytes.IndexByte
, the best way to find the first zero byte.
Only use for performance tuning.
package main
import (
"fmt"
"reflect"
"unsafe"
)
func BytesToString(b []byte) string {
return *(*string)(unsafe.Pointer(&b))
}
func StringToBytes(s string) []byte {
return *(*[]byte)(unsafe.Pointer(&s))
}
func main() {
b := []byte{'b', 'y', 't', 'e'}
s := BytesToString(b)
fmt.Println(s)
b = StringToBytes(s)
fmt.Println(string(b))
}
string
may have serious implications if later the byte slice is modified. string
values in Go are defined to be immutable, to which the entire Go runtime and libraries build on. You will teleport yourself into the middle of the most mysterious bugs and runtime errors if you go down this path.
Use this:
bytes.NewBuffer(byteArray).String()
byteArray[:]
since bytes.NewBuffer
takes a []byte
; b) the question said the array has trailing zeros that you don't deal with; c) if instead your variable is a []byte
(the only way your line will compile) then your line is just a slow way of doing string(v)
.
Though not extremely performant, the only readable solution is:
// Split by separator and pick the first one.
// This has all the characters till null, excluding null itself.
retByteArray := bytes.Split(byteArray[:], []byte{0}) [0]
// OR
// If you want a true C-like string, including the null character
retByteArray := bytes.SplitAfter(byteArray[:], []byte{0}) [0]
A full example to have a C-style byte array:
package main
import (
"bytes"
"fmt"
)
func main() {
var byteArray = [6]byte{97,98,0,100,0,99}
cStyleString := bytes.SplitAfter(byteArray[:], []byte{0}) [0]
fmt.Println(cStyleString)
}
A full example to have a Go style string excluding the nulls:
package main
import (
"bytes"
"fmt"
)
func main() {
var byteArray = [6]byte{97, 98, 0, 100, 0, 99}
goStyleString := string(bytes.Split(byteArray[:], []byte{0}) [0])
fmt.Println(goStyleString)
}
This allocates a slice of slice of bytes. So keep an eye on performance if it is used heavily or repeatedly.
Use slices instead of arrays for reading. For example, io.Reader accepts a slice, not an array.
Use slicing instead of zero padding.
Example:
buf := make([]byte, 100)
n, err := myReader.Read(buf)
if n == 0 && err != nil {
log.Fatal(err)
}
consume(buf[:n]) // consume() will see an exact (not padded) slice of read data
s := a[:n]
or s := string(a[:n])
if you need a string. If n
is not directly available it must be computed, e.g. by looking for a specific/zero byte in the buffer (array) as Daniel suggests.
Here is an option that removes the null bytes:
package main
import "golang.org/x/sys/windows"
func main() {
b := []byte{'M', 'a', 'r', 'c', 'h', 0}
s := windows.ByteSliceToString(b)
println(s == "March")
}
https://pkg.go.dev/golang.org/x/sys/unix#ByteSliceToString
https://pkg.go.dev/golang.org/x/sys/windows#ByteSliceToString
build constraints exclude all Go files in...
Success story sharing
bytes.IndexByte()
which searches for a singlebyte
instead ofbytes.Index()
with a byte slice containing 1 byte.