Details
-
Improvement
-
Resolution: Fixed
-
Major
-
0.6.2
-
None
-
Unknown
-
N/A
-
N/A
-
Description
In particular when, e.g., the actual embedding instantly fails or chunks don't need to be re-embedded, it is noticeable that indexing is quite slow. I found that this is due to two reasons:
- The regular expression for finding the last heading at the end of a chunk is surprisingly slow.
- The Solr index is committed very frequently (after every document).
Both can easily be improved.