TrueCloudLab/tzhash
15
0
Fork 6
Code Issues 3 Pull requests Releases 2 Activity
tzhash/tz/avx_amd64.s

108 lines
2.6 KiB
ArmAsm
Raw Normal View History

Initial
2018-12-29 13:04:17 +00:00
#include "textflag.h"
Use macros in AVX hash implementation
2019-10-10 08:29:40 +00:00
// mul2 multiplicates FROM by 2, stores result in R1
// and uses R1, R2 and R3 for internal computations.
#define mul2(FROM, TO, R2, R3) \
VPSLLQ $1, FROM, TO \
VPALIGNR $8, TO, FROM, R2 \
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VPSRLQ $63, R2, R2 \
Optimize AVX implementation 1. Do the same mask trick as with AVX2. 2. Get rid of load, generate constant on the fly. ``` name old time/op new time/op delta Sum/AVXInline_digest-8 2.26ms ± 4% 2.17ms ± 5% -4.05% (p=0.000 n=19+17) name old speed new speed delta Sum/AVXInline_digest-8 44.3MB/s ± 4% 46.2MB/s ± 5% +4.25% (p=0.000 n=19+17) ``` Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-29 12:56:02 +00:00
VANDPD TO, X14, R3 \
Use macros in AVX hash implementation
2019-10-10 08:29:40 +00:00
VPUNPCKHQDQ R3, R3, R3 \
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VXORPD R2, TO, TO \
VXORPD R3, TO, TO
Use macros in AVX hash implementation
2019-10-10 08:29:40 +00:00
Optimize AVX implementation 1. Do the same mask trick as with AVX2. 2. Get rid of load, generate constant on the fly. ``` name old time/op new time/op delta Sum/AVXInline_digest-8 2.26ms ± 4% 2.17ms ± 5% -4.05% (p=0.000 n=19+17) name old speed new speed delta Sum/AVXInline_digest-8 44.3MB/s ± 4% 46.2MB/s ± 5% +4.25% (p=0.000 n=19+17) ``` Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-29 12:56:02 +00:00
#define mask(bit, tmp, to) \
VPSRLW bit, X10, tmp \
VPAND X12, tmp, to \ // to = 0x000<bit>000<bit>...
VPSUBW to, X13, to // to = 0xFFFF.. or 0x0000 depending on bit
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
#define mulBit(bit) \
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VMOVDQU X0, X8 \
VMOVDQU X2, X9 \
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
mul2(X0, X5, X6, X7) \
VXORPD X1, X5, X0 \
mul2(X2, X5, X6, X7) \
VXORPD X3, X5, X2 \
Optimize AVX implementation 1. Do the same mask trick as with AVX2. 2. Get rid of load, generate constant on the fly. ``` name old time/op new time/op delta Sum/AVXInline_digest-8 2.26ms ± 4% 2.17ms ± 5% -4.05% (p=0.000 n=19+17) name old speed new speed delta Sum/AVXInline_digest-8 44.3MB/s ± 4% 46.2MB/s ± 5% +4.25% (p=0.000 n=19+17) ``` Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-29 12:56:02 +00:00
mask(bit, X6, X5) \
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
VANDPD X0, X5, X1 \
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VXORPD X8, X1, X1 \
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
VANDPD X2, X5, X3 \
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VXORPD X9, X3, X3
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
Initial
2018-12-29 13:04:17 +00:00
// func mulBitRight(c00, c01, c10, c11, e *[2]uint64)
TEXT ·mulBitRight(SB),NOSPLIT,$0
MOVQ c00+0(FP), AX
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VMOVDQU (AX), X0
VMOVDQU X0, X8 // remember c00 value
Initial
2018-12-29 13:04:17 +00:00
MOVQ c01+8(FP), BX
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VMOVDQU (BX), X1
Initial
2018-12-29 13:04:17 +00:00
MOVQ c10+16(FP), CX
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VMOVDQU (CX), X2
VMOVDQU X2, X9 // remember c10 value
Initial
2018-12-29 13:04:17 +00:00
MOVQ c11+24(FP), DX
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VMOVDQU (DX), X3
Initial
2018-12-29 13:04:17 +00:00
Optimize AVX implementation 1. Do the same mask trick as with AVX2. 2. Get rid of load, generate constant on the fly. ``` name old time/op new time/op delta Sum/AVXInline_digest-8 2.26ms ± 4% 2.17ms ± 5% -4.05% (p=0.000 n=19+17) name old speed new speed delta Sum/AVXInline_digest-8 44.3MB/s ± 4% 46.2MB/s ± 5% +4.25% (p=0.000 n=19+17) ``` Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-29 12:56:02 +00:00
VPXOR X13, X13, X13 // Y13 = 0x0000...
VPCMPEQB X14, X14, X14 // Y14 = 0xFFFF...
VPSUBQ X14, X13, X13
VPSLLQ $63, X13, X14
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
mul2(X0, X5, X6, X7) // c00 *= 2
VXORPD X5, X1, X0 // c00 += c01
mul2(X2, X5, X6, X7) // c10 *= 2
VXORPD X3, X5, X2 // c10 += c11
MOVQ e+32(FP), CX
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VMOVDQU (CX), X5
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
VANDPD X0, X5, X1 // c01 = c00 + e
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VXORPD X8, X1, X1 // c01 += X8 (old c00)
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
VANDPD X2, X5, X3 // c11 = c10 + e
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VXORPD X9, X3, X3 // c11 += x9 (old c10)
Initial
2018-12-29 13:04:17 +00:00
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VMOVDQU X0, (AX)
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
MOVQ c10+16(FP), CX
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VMOVDQU X2, (CX)
VMOVDQU X1, (BX)
VMOVDQU X3, (DX)
Initial
2018-12-29 13:04:17 +00:00
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
RET
Initial
2018-12-29 13:04:17 +00:00
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
TEXT ·mulByteRight(SB),NOSPLIT,$0
MOVQ c00+0(FP), AX
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VMOVDQU (AX), X0
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
MOVQ c01+8(FP), BX
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VMOVDQU (BX), X1
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
MOVQ c10+16(FP), CX
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VMOVDQU (CX), X2
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
MOVQ c11+24(FP), DX
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VMOVDQU (DX), X3
Optimize AVX implementation 1. Do the same mask trick as with AVX2. 2. Get rid of load, generate constant on the fly. ``` name old time/op new time/op delta Sum/AVXInline_digest-8 2.26ms ± 4% 2.17ms ± 5% -4.05% (p=0.000 n=19+17) name old speed new speed delta Sum/AVXInline_digest-8 44.3MB/s ± 4% 46.2MB/s ± 5% +4.25% (p=0.000 n=19+17) ``` Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-29 12:56:02 +00:00
MOVQ $0, CX
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
MOVB b+32(FP), CX
Initial
2018-12-29 13:04:17 +00:00
Optimize AVX implementation 1. Do the same mask trick as with AVX2. 2. Get rid of load, generate constant on the fly. ``` name old time/op new time/op delta Sum/AVXInline_digest-8 2.26ms ± 4% 2.17ms ± 5% -4.05% (p=0.000 n=19+17) name old speed new speed delta Sum/AVXInline_digest-8 44.3MB/s ± 4% 46.2MB/s ± 5% +4.25% (p=0.000 n=19+17) ``` Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-29 12:56:02 +00:00
VPXOR X13, X13, X13 // X13 = 0x0000...
VPCMPEQB X14, X14, X14 // X14 = 0xFFFF...
VPSUBQ X14, X13, X10
VPSUBW X14, X13, X12 // X12 = 0x00010001... (packed words of 1)
VPSLLQ $63, X10, X14 // X14 = 0x10000000... (packed quad-words with HSB set)
MOVQ CX, X10
VPSHUFLW $0, X10, X11
VPSHUFD $0, X11, X10
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
mulBit($7)
mulBit($6)
mulBit($5)
mulBit($4)
mulBit($3)
mulBit($2)
mulBit($1)
mulBit($0)
Initial
2018-12-29 13:04:17 +00:00
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VMOVDQU X0, (AX)
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
MOVQ c10+16(FP), CX
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VMOVDQU X2, (CX)
VMOVDQU X1, (BX)
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
MOVQ c11+24(FP), DX
Replace all SSE instructions with AVX ones Also use integer MOV* variant instead of floating-point one. Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
2021-12-28 12:24:46 +00:00
VMOVDQU X3, (DX)
Make use of AVX2 in Sum() by default
2019-06-21 15:47:01 +00:00
Add AVX implementation with inlined multiplication Perform multiplication by-byte instead of by-bit as in AVX2Inline implementation.
2019-10-15 11:37:59 +00:00
RET
Reference in a new issue Copy permalink