Benchmarking random access in Lance Test setup Creating dataset Benchmarking Take Key lookup Duckdb integration Conclusion

-

sudo lshw -short
Class Description
=============================================================
system 20M9CTO1WW (LENOVO_MT_20M9_BU_Think_FM_ThinkPad P52)
memory 128GiB System Memory
memory 32GiB SODIMM DDR4 Synchronous 2667 MHz (0.4 ns)
memory 32GiB SODIMM DDR4 Synchronous 2667 MHz (0.4 ns)
memory 32GiB SODIMM DDR4 Synchronous 2667 MHz (0.4 ns)
memory 32GiB SODIMM DDR4 Synchronous 2667 MHz (0.4 ns)
memory 384KiB L1 cache
memory 1536KiB L2 cache
memory 12MiB L3 cache
processor Intel(R) Xeon(R) E-2176M CPU @ 2.70GHz
storage Samsung SSD 980 PRO 2TB
pa.dataset.write_dataset(lance_dataset.scanner().to_reader(), 
"take.parquet",
format="parquet")
Lance: mean time per secret is 0.0006225714343229975
Parquet: mean time per secret is 1.246656603929473
throughput is computed as “1 / mean time per key”

ASK DUKE

What are your thoughts on this topic?
Let us know in the comments below.

2 COMMENTS

0 0 votes
Article Rating
guest
2 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments

Share this article

Recent posts

2
0
Would love your thoughts, please comment.x
()
x