Optimize encodeUtf8 by checking if the byte array is already pinned #471

Activity

1. Bodigrim commented on Sep 29, 2022

   

   Bodigrim

   on Sep 29, 2022

   Contributor

   More actions

   text/src/Data/Text/Encoding.hs

   Lines 528 to 538 in 5558730

   	encodeUtf8 :: Text -> ByteString
   	encodeUtf8 (Text arr off len)
   	| len == 0 = B.empty
   	-- It would be easier to use Data.ByteString.Short.fromShort and slice later,
   	-- but this is undesirable when len is significantly smaller than length arr.
   	| otherwise = unsafeDupablePerformIO $ do
   	marr@(A.MutableByteArray mba) <- unsafeSTToIO $ A.newPinned len
   	unsafeSTToIO $ A.copyI len marr 0 arr off
   	let fp = ForeignPtr (byteArrayContents# (unsafeCoerce# mba))
   	(PlainPtr mba)
   	pure $ B.fromForeignPtr fp 0 len

   Might be worth a try, but take care not to retain the original pinned ByteArray if Text has been sliced.
2. noughtmare commented on Sep 30, 2022

   

   noughtmare

   on Sep 30, 2022

   Author

   More actions

   Ah, I didn't think about slicing. I guess it is best to only reuse the original byte array if it is both pinned and not sliced. Although perhaps it would be acceptable to reuse the original byte array if the slice is still like 90% of the original size, but I don't think that is particularly common.
3. Lysxia

   added

   feature request

   on Feb 4, 2023