chore(deps): update dependency pyparsing to v3 - #39
Open
manifest-self-hosted-renovate[bot] wants to merge 1 commit into
Open
chore(deps): update dependency pyparsing to v3#39manifest-self-hosted-renovate[bot] wants to merge 1 commit into
manifest-self-hosted-renovate[bot] wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
==2.4.6→==3.3.2Release Notes
pyparsing/pyparsing (pyparsing)
v3.3.2Compare Source
Defined pyparsing-specific warning classes so that they can be selectively enabled
or disabled without affecting warnings raised by other libraries in the same Python
app:
PyparsingWarning- base warning for all pyparsing-specific warnings (inheritsfrom
UserWarning)PyparsingDeprecationWarning- warning for using deprecated features (inheritsfrom
PyparsingWarningandDeprecationWarning)PyparsingDiagnosticWarning- warning raised when pyparsing diagnostics areenabled and a diagnostic feature is used (inherits from
PyparsingWarning)Added
as_datetimeparse action topyparsing.common- a more generalizedversion of the
convert_to_datetimeparse action (supports any expression that extractsdate/time fields into "year", "month", "day", etc. results names), and validates
that the parsed fields represent a valid date and time.
Added
iso8601_date_validatedandiso8601_datetime_validatedexpressions topyparsing.common, which return a Pythondatetime.datetimeVarious performance improvements in
ParseResultsclass and core functions, with10-20% performance overall.
Added
regex_inverterweb page (using PyScript) to demonstrate using theinv_regex.pyexample.
Expanded regex forms handled by the
examples/inv_regex.pyexample:?P<name>){m,}and{,n})[^...])Added
SPy(Simplified Python) parser to examples.v3.3.1Compare Source
Marc Mueller for submitted issue and PR. Fixes #626.
v3.3.0Compare Source
===========================================================================================
The version 3.3.0 release will begin emitting
DeprecationWarningsfor pyparsing methodsthat have been renamed to PEP8-compliant names (introduced in pyparsing 3.0.0, in August,
2021, with legacy names retained as aliases). In preparation, I added in pyparsing
3.2.2 a utility for finding and replacing the legacy method names with the new names.
This utility is located at
pyparsing/tools/cvt_pep8_names.py. This script will scan allPython files specified on the command line, and if the
-uoption is selected, willreplace all occurrences of the old method names with the new PEP8-compliant names,
updating the files in place.
Here is an example that converts all the files in the pyparsing
/examplesdirectory:v3.2.5Compare Source
Wordexpressions that include a spacecharacter, if that expression was then copied, either directly with .copy() or
by adding a results name, or included in another construct (like
DelimitedList)that makes a copy internally. Issue #618, reported by mstinberg, among others -
thanks, and sorry for the inconvenience.
v3.2.4Compare Source
Barring any catastrophic bugs in this release, this will be the last release in
the 3.2.x line. The next release, 3.3.0, will begin emitting
DeprecationWarningswhen the pre-PEP8 methods are used (see header notes above for more information,
including available automation for converting any existing code using
pyparsing with the old names).
Fixed bug when using a copy of a
Wordexpression (either by using the explicitcopy()method, or attaching a results name), and setting a new expression name,a raised
ParseExceptionstill used the original expression name. Also affectedRegexexpressions withas_matchoras_group_list= True. Reported byWaqas Ilyas, in Issue #612 - good catch!
Fixed type annotation for
replace_with, to acceptAnytype. Fixes Issue #602,reported by esquonk.
Added locking around potential race condition in
ParserElement.reset_cache, aswell as other cache-related methods. Fixes Issue #604, reported by CarlosDescalziIM.
Substantial update to docstrings and doc generation in preparation for 3.3.0,
great effort by FeRD, thanks!
Notable addition by FeRD to convert docstring examples to work with doctest! This
was long overdue, thanks so much!
v3.2.3Compare Source
nested_exprcould overwrite parse actionsfor defined content, and could truncate list of items within a nested list.
Fixes Issue #600, reported by hoxbro and luisglft, with helpful diag logs and
repro code.
v3.2.2Compare Source
Released
cvt_pyparsing_pep8_names.pyconversion utility to upgrade pyparsing-basedprograms and libraries that use legacy camelCase names to use the new PEP8-compliant
snake_case method names. The converter can also be imported into other scripts as
Fixed bug in
nested_exprwhere nested contents were stripped of whitespace whenthe default whitespace characters were cleared (raised in this StackOverflow
question https://stackoverflow.com/questions/79327649 by Ben Alan). Also addressed
bug in resolving PEP8 compliant argument name and legacy argument name.
Fixed bug in
rest_of_lineand the underlyingRegexclass, in which matching apattern that could match an empty string (such as
".*"or"[A-Z]*"would not raisea
ParseExceptionat or beyond the end of the input string. This could cause aninfinite parsing loop when parsing
rest_of_lineat the end of the input string.Reported by user Kylotan, thanks! (Issue #593)
Enhancements and extra input validation for
pyparsing.util.make_compressed_re- seeusage in
examples/complex_chemical_formulas.pyand result in the generated railroaddiagram
examples/complex_chemical_formulas_diagram.html. Properly escapes characterslike "." and "*" that have special meaning in regular expressions.
Fixed bug in
one_of()to properly escape characters that are regular expression markers(such as '*', '+', '?', etc.) before building the internal regex.
Better exception message for
MatchFirstandOrexpressions, showing all alternativesrather than just the first one. Fixes Issue #592, reported by Focke, thanks!
Added return type annotation of "-> None" for all
__init__()methods, to satisfymypy --stricttype checking. PR submitted by FeRD, thank you!Added optional argument
show_hiddentocreate_diagramto showelements that are used internally by pyparsing, but are not part of the actual
parser grammar. For instance, the
Tagclass can insert values into the parsedresults but it does not actually parse any input, so by default it is not included
in a railroad diagram. By calling
create_diagramwithshow_hidden = True,these internal elements will be included. (You can see this in the tag_metadata.py
script in the examples directory.)
Fixed bug in
number_words.pyexample. Also addedebnf_number_words.pyto demonstrateusing the
ebnf.pyEBNF parser generator to build a similar parser directly fromEBNF.
Fixed syntax warning raised in
bigquery_view_parser.py, invalid escape sequence "\s".Reported by sameer-google, nice catch! (Issue #598)
Added support for Python 3.14.
v3.2.1Compare Source
Updated generated railroad diagrams to make non-terminal elements links to their related
sub-diagrams. This greatly improves navigation of the diagram, especially for
large, complex parsers.
Simplified railroad diagrams emitted for parsers using
infix_notation, by hidinglookahead terms. Renamed internally generated expressions for clarity, and improved
diagramming.
Improved performance of
cpp_style_comment,c_style_comment,common.fnumberand
common.ieee_floatRegexexpressions. PRs submitted by Gabriel Gerlero,nice work, thanks!
Add missing type annotations to
match_only_at_col,replace_with,remove_quotes,with_attribute, andwith_class. Issue #585 reported by rafrafrek.Added generated diagrams for many of the examples.
Replaced old
examples/0README.htmlfile withexamples/README.mdfile.v3.2.0Compare Source
Discontinued support for Python 3.6, 3.7, and 3.8. Adopted new Python features from
Python versions 3.7-3.9:
imported from the
typingmodule (e.g.,list[str]vsList[str]).in dicts (including removal of uses of
OrderedDict).pdb.set_trace()call inParserElement.set_break()tobreakpoint().typing.NamedTupletodataclasses.dataclassin railroad diagrammingcode.
from __future__ import annotationsto clean up some type annotations.(with assistance from ISyncWithFoo, issue #535, thanks for the help!)
POSSIBLE BREAKING CHANGES
The following bugfixes may result in subtle changes in the results returned or
exceptions raised by pyparsing.
Fixed code in
ParseElementEnhancesubclasses thatreplaced detailed exception messages raised in contained expressions with a
less-specific and less-informative generic exception message and location.
If your code has conditional logic based on the message content in raised
ParseExceptions, this bugfix may require changes in your code.Fixed bug in
transform_string()where whitespacein the input string was not properly preserved in the output string.
If your code uses
transform_string, this bugfix may require changes inyour code.
Fixed bug where an
IndexErrorraised in a parse action wasincorrectly handled as an
IndexErrorraised as part of theParserElementparsing methods, and reraised as a
ParseException. Now anIndexErrorthat raises inside a parse action will properly propagate out as an
IndexError.(Issue #573, reported by August Karlstedt, thanks!)
If your code raises
IndexErrors in parse actions, this bugfix may requirechanges in your code.
FIXES AND NEW FEATURES
Added type annotations to remainder of
pyparsingpackage, and addedmypyrun to
tox.ini, so that type annotations are now run as part of pyparsing's CI.Addresses Issue #373, raised by Iwan Aucamp, thanks!
Exception message format can now be customized, by overriding
ParseBaseException.format_message:(PR #571 submitted by Odysseyas Krystalakos, nice work!)
run_testsnow detects if an exception is raised in a parse action, and willreport it with an enhanced error message, with the exception type, string,
and parse action name.
QuotedStringnow handles translation of escaped integer, hex, octal, andUnicode sequences to their corresponding characters.
Fixed the displayed output of
Regexterms to deduplicate repeated backslashes,for easier reading in debugging, printing, and railroad diagrams.
Fixed (or at least reduced) elusive bug when generating railroad diagrams,
where some diagram elements were just empty blocks. Fix submitted by RoDuth,
thanks a ton!
Fixed railroad diagrams that get generated with a parser containing a
Regexelementdefined using a verbose pattern - the pattern gets flattened and comments removed
before creating the corresponding diagram element.
Defined a more performant regular expression used internally by
common_html_entity.Regexinstances can now be created using a callable that takes no argumentsand just returns a string or a compiled regular expression, so that creating complex
regular expression patterns can be deferred until they are actually used for the first
time in the parser.
Added optional
flattenBoolean argument toParseResults.as_list(), toreturn the parsed values in a flattened list.
Added
indentandbase_1arguments topyparsing.testing.with_line_numbers. Whenusing
with_line_numbersinside a parse action, setbase_1=False, since thereported
locvalue is 0-based.indentcan be a leading string (typically ofspaces or tabs) to indent the numbered string passed to
with_line_numbers.Added while working on #557, reported by Bernd Wechner.
NEW/ENHANCED EXAMPLES
Added query syntax to
mongodb_query_expression.pywith:"contains any", and "contains none")
and "=~" operator to support regex matching
a[0]style array referencingAdded
lox_parser.pyexample, a parser for the Lox language used as a tutorial inRobert Nystrom's "Crafting Interpreters" (http://craftinginterpreters.com/).
With helpful corrections from RoDuth.
Added
complex_chemical_formulas.pyexample, to add parsing capability forformulas such as "3(C₆H₅OH)₂".
Updated
tag_emitter.pyto use newTagclass, introduced in pyparsing3.1.3.
v3.1.4Compare Source
referenced
re.Pattern. Since this type was introduced in Python 3.7, using this typedefinition broke Python 3.6 installs of pyparsing 3.1.3. PR submitted by Felix Fontein,
nice work!
v3.1.3Compare Source
Added new
TagParserElement, for inserting metadata into the parsed results.This allows a parser to add metadata or annotations to the parsed tokens.
The
Tagelement also accepts an optionalvalueparameter, defaulting toTrue.See the new
tag_metadata.pyexample in theexamplesdirectory.Example:
add tag indicating mood
prints:
Added example
mongodb_query_expression.py, to convert human-readable infix queryexpressions (such as
a==100 and b>=200) and transform them into the equivalentquery argument for the pymongo package (
{'$and': [{'a': 100}, {'b': {'$gte': 200}}]}).Supports many equality and inequality operators - see the docstring for the
transform_queryfunction for more examples.Fixed issue where PEP8 compatibility names for
ParserElementstatic methods werenot themselves defined as
staticmethods. When called using aParserElementinstance,this resulted in a
TypeErrorexception. Reported by eylenburg (#548).To address a compatibility issue in RDFLib, added a property setter for the
ParserElement.nameproperty, to callParserElement.set_name.Modified
ParserElement.set_name()to accept a None value, to clear the definedname and corresponding error message for a
ParserElement.Updated railroad diagram generation for
ZeroOrMoreandOneOrMoreexpressions withstop_onexpressions, while investigating #558, reported by user Gu_f.Added
<META>tag to HTML generated for railroad diagrams to force UTF-8 encodingwith older browsers, to better display Unicode parser characters.
Fixed some cosmetics/bugs in railroad diagrams:
show_groups=Falseshow_results_names=TrueSome type annotations added for parse action related methods, thanks August
Karlstedt (#551).
Added exception type to
trace_parse_actionexception output, while investigatingSO question posted by medihack.
Added
set_namecalls to internal expressions generated ininfix_notation, forimproved railroad diagramming.
delta_time,lua_parser,decaf_parser, androman_numeralsexamples cleaned upto use latest PEP8 names and add minor enhancements.
Fixed bug (and corresponding test code) in
delta_timeexample that did not handleweekday references in time expressions (like "Monday at 4pm") when the weekday was
the same as the current weekday.
Minor performance speedup in
trim_arity, to benefit any parsers using parse actions.Added early testing support for Python 3.13 with JIT enabled.
v3.1.2Compare Source
Added
ieee_floatexpression topyparsing.common, which parses float values,plus "NaN", "Inf", "Infinity". PR submitted by Bob Peterson (#538).
Updated pep8 synonym wrappers for better type checking compatibility. PR submitted
by Ricardo Coccioli (#507).
Fixed empty error message bug, PR submitted by InSync (#534). This should return
pyparsing's exception messages to a former, more helpful form. If you have code that
parses the exception messages returned by pyparsing, this may require some code
changes.
Added unit tests to test for exception message contents, with enhancement to
pyparsing.testing.assertRaisesParseExceptionto accept an expected exception message.Updated example
select_parser.pyto use PEP8 names and added Groups for better retrievalof parsed values from multiple SELECT clauses.
Added example
email_address_parser.py, as suggested by John Byrd (#539).Added example
directx_x_file_parser.pyto parse DirectX template definitions, andgenerate a Pyparsing parser from a template to parse .x files.
Some code refactoring to reduce code nesting, PRs submitted by InSync.
All internal string expressions using '%' string interpolation and
str.format()converted to f-strings.
v3.1.1Fixed regression in Word(min), reported by Ricardo Coccioli, good catch! (Issue #502)
Fixed bug in bad exception messages raised by Forward expressions. PR submitted
by Kyle Sunden, thanks for your patience and collaboration on this (#493).
Fixed regression in SkipTo, where ignored expressions were not checked when looking
for the target expression. Reported by catcombo, Issue #500.
Fixed type annotation for enable_packrat, PR submitted by Mike Urbach, thanks! (Issue #498)
Some general internal code cleanup. (Instigated by Michal Čihař, Issue #488)
v3.1.0tag_emitter.pyto examples. This example demonstrates how to inserttags into your parsed results that are not part of the original parsed text.
v3.0.9Compare Source
Added Unicode set
BasicMultilingualPlane(may also be referencedas
BMP) representing the Basic Multilingual Plane (Unicodecharacters up to code point 65535). Can be used to parse
most language characters, but omits emojis, wingdings, etc.
Raised in discussion with Dave Tapley (issue #392).
To address mypy confusion of
pyparsing.Optionalandtyping.Optionalresulting in
error: "_SpecialForm" not callablemessagereported in issue #365, fixed the import in
exceptions.py. Nicesleuthing by Iwan Aucamp and Dominic Davis-Foster, thank you!
(Removed definitions of
OptionalType,DictType, andIterableTypeand replaced them with
typing.Optional,typing.Dict, andtyping.Iterablethroughout.)Fixed typo in jinja2 template for railroad diagrams, thanks for the
catch Nioub (issue #388).
Removed use of deprecated
pkg_resourcespackage inrailroad diagramming code (issue #391).
Updated
bigquery_view_parser.pyexample to parse examples athttps://cloud.google.com/bigquery/docs/reference/legacy-sql
v3.0.8Compare Source
API CHANGE: modified
pyproject.tomlto require Python version3.6.8 or later for pyparsing 3.x. Earlier minor versions of 3.6
fail in evaluating the
version_infoclass (implemented usingtyping.NamedTuple). If you are using an earlier version of Python3.6, you will need to use pyparsing 2.4.7.
Improved pyparsing import time by deferring regex pattern compiles.
PR submitted by Anthony Sottile to fix issue #362, thanks!
Updated build to use flit, PR by Michał Górny, added
BUILDING.mddoc and removed old Windows build scripts - nice cleanup work!
More type-hinting added for all arithmetic and logical operator
methods in
ParserElement. PR from Kazantcev Andrey, thank you.Fixed
infix_notation's definitions oflparandrpar, to acceptparse expressions such that they do not get suppressed in the parsed
results. PR submitted by Philippe Prados, nice work.
Fixed bug in railroad diagramming with expressions containing
Combineelements. Reported by Jeremy White, thanks!
Added
show_groupsargument tocreate_diagramto highlight groupedelements with an unlabeled bounding box.
Added
unicode_denormalizer.pyto the examples as a demonstrationof how Python's interpreter will accept Unicode characters in
identifiers, but normalizes them back to ASCII so that identifiers
printand𝕡𝓻ᵢ𝓃𝘁and𝖕𝒓𝗂𝑛ᵗare all equivalent.Removed imports of deprecated
sre_constantsmodule for catchingexceptions when compiling regular expressions. PR submitted by
Serhiy Storchaka, thank you.
v3.0.7Compare Source
Fixed bug #345, in which delimitedList changed expressions in place
using
expr.streamline(). Reported by Kim Gräsman, thanks!Fixed bug #346, when a string of word characters was passed to WordStart
or
WordEndinstead of just taking the default value. Originally postedas a question by Parag on StackOverflow, good catch!
Fixed bug #350, in which
Whiteexpressions could fail to match due tounintended whitespace-skipping. Reported by Fu Hanxi, thank you!
Fixed bug #355, when a
QuotedStringis defined with characters in itsquoteChar string containing regex-significant characters such as ., *,
?, [, ], etc.
Fixed bug in
ParserElement.run_testswhere comments would be displayedusing
with_line_numbers.Added optional "min" and "max" arguments to
delimited_list. PRsubmitted by Marius, thanks!
Added new API change note in
whats_new_in_pyparsing_3_0_0, regardinga bug fix in the
bool()behavior ofParseResults.Prior to pyparsing 3.0.x, the
ParseResultsclass implementation of__bool__would returnFalseif theParseResultsitem list was empty,even if it contained named results. In 3.0.0 and later,
ParseResultswillreturn
Trueif either the item list is not empty or if the namedresults dict is not empty.
generate an empty ParseResults by parsing a blank string with
a ZeroOrMore
add a results name to the result
Prints:
In previous versions, the second call to
bool()would returnFalse.Minor enhancement to Word generation of internal regular expression, to
emit consecutive characters in range, such as "ab", as "ab", not "a-b".
Fixed character ranges for search terms using non-Western characters
in booleansearchparser, PR submitted by tc-yu, nice work!
Additional type annotations on public methods.
v3.0.6Compare Source
Added
suppress_warning()method to individually suppress a warning on aspecific ParserElement. Used to refactor
original_text_forto preserveinternal results names, which, while undocumented, had been adopted by
some projects.
Fix bug when
delimited_listwas called with a str literal instead of aparse expression.
v3.0.5Compare Source
Added return type annotations for
col,line, andlineno.Fixed bug when
warn_ungrouped_named_tokens_in_collectionwarning was raisedwhen assigning a results name to an
original_text_forexpression.(Issue #110, would raise warning in packaging.)
Fixed internal bug where
ParserElement.streamline()would not return self ifalready streamlined.
Changed
run_tests()output to default to not showing line and column numbers.If line numbering is desired, call with
with_line_numbers=True. Also fixedminor bug where separating line was not included after a test failure.
v3.0.4Compare Source
Fixed bug in which
Dictclasses did not correctly return tokens as nestedParseResults, reported by and fix identified by Bu Sun Kim, many thanks!!!Documented API-changing side-effect of converting
ParseResultsto use__slots__to pre-define instance attributes. This means that code written like this (which
was allowed in pyparsing 2.4.7):
result = Word(alphas).parseString("abc")
result.xyz = 100
now raises this Python exception:
AttributeError: 'ParseResults' object has no attribute 'xyz'
To add new attribute values to ParseResults object in 3.0.0 and later, you must
assign them using indexed notation:
result["xyz"] = 100
You will still be able to access this new value as an attribute or as an
indexed item.
Fixed bug in railroad diagramming where the vertical limit would count all
expressions in a group, not just those that would create visible railroad
elements.
v3.0.3Compare Source
Fixed regex typo in
one_offix foras_keyword=True.Fixed a whitespace-skipping bug, Issue #319, introduced as part of the revert
of the
LineStartchanges. Reported by Marc-Alexandre Côté,thanks!
Added header column labeling > 100 in
with_line_numbers- some input linesare longer than others.
v3.0.2Compare Source
Reverted change in behavior with
LineStartandStringStart, which changed theinterpretation of when and how
LineStartandStringStartshould match whena line starts with spaces. In 3.0.0, the
xxxStartexpressions were notreally treated like expressions in their own right, but as modifiers to the
following expression when used like
LineStart() + expr, so that if therewere whitespace on the line before
expr(which would match in versions priorto 3.0.0), the match would fail.
3.0.0 implemented this by automatically promoting
LineStart() + exprtoAtLineStart(expr), which broke existing parsers that did not expectexprtonecessarily be right at the start of the line, but only be the first token
found on the line. This was reported as a regression in Issue #317.
In 3.0.2, pyparsing reverts to the previous behavior, but will retain the new
AtLineStartandAtStringStartexpression classes, so that parsers can chosewhichever behavior applies in their specific instance. Specifically:
matches expr if it is the first token on the line
(allows for leading whitespace)
matches only if expr is found in column 1
Performance enhancement to
one_ofto always generate an internalRegex,even if
caselessoras_keywordargs are given asTrue(unless explicitlydisabled by passing
use_regex=False).IndentedBlockclass now works withrecursiveflag. By default, theresults parsed by an
IndentedBlockare grouped. This can be disabled by constructingthe
IndentedBlockwithgrouped=False.v3.0.1Compare Source
Fixed bug where
Word(max=n)did not match word groups less than length 'n'.Thanks to Joachim Metz for catching this!
Fixed bug where
ParseResultsaccidentally created recursive contents.Joachim Metz on this one also!
Fixed bug where
warn_on_multiple_string_args_to_oneofwarning is raisedeven when not enabled.
v3.0.0Compare Source
===========================================================================================
Deprecated
indentedBlock, when converted using thecvt_pyparsing_pep8_namesutility, will emit
UserWarningsthat additional code changes will be required.This is because the new
IndentedBlockclass no longer requires the calling codeto supply an indent stack, while adding support for nested indentation levels
and grouping.
Deprecated
locatedExpr, when converted using thecvt_pyparsing_pep8_namesutility, will emit
UserWarningsthat additional code changes may be required.The new
Locatedclass removes the extra grouping level of the parsed values.(If the original
locatedExprparser was defined with a results name, thenthe extra grouping is retained, so that the results name nesting works properly;
in this case, no code changes would be required.)
Updated all examples and test cases to use PEP8 names (unless the test case is specifically
designed to test behavior of a legacy method). Added railroad diagrams for some examples.
Added exception handling when calling
formatted_message(), so thatstr(exception)always returns at least something.
All unit tests pass with Python 3.14, including 3.14t. This does not necessarily
mean that pyparsing is now thread-safe, just that when run in the free-threaded
interpreter, there were no errors. None of the unit tests try to do any parsing
with multiple threads - they test the basic functionality of the library, under various
versions of packrat and left-recursive parsing.
Added AI instructions so that AI agents can be prompted with best practices
for generating parsers using pyparsing code. These instructions are in the
ai/best_practices.mdfile, and can be accessed programmatically by callingpyparsing.show_best_practices()or runningpython -m pyparsing.ai.show_best_practicesfrom the command line, after installing the
pyparsingpackage.Implemented a TINY language parser/interpreter using pyparsing, in the
examples/tinydirectory. This is a little tutorial language that I used to demonstrate how to use pyparsing to
build a simple interpreter, following a recommended parser+AST+engine+run structure.
The
docssub-directory also includes transcripts of the AI session used to create theparser and the interpreter. The
samplessub-directory includes a few sample TINY programs.Fixed minor formatting bugs in
pyparsing.testing.with_line_numbers, found during developmentof the TINY language example.
Added test in
DelimitedListandnested_exprwhich auto-suppress delimiting commas toavoid wrapping in a
Suppressif delimiter is already aSuppress.Added performance benchmarking tools and documentation:
tests/perf_pyparsing.pyruns a series of benchmark parsing tests, exercising differentaspects of the pyparsing package. For cross-version analysis, this script can export
results as CSV and append to a consolidated data file.
run_perf_all_tags.bat(Windows) andrun_perf_all_tags.sh(Ubuntu/bash)execute the benchmark across multiple Python versions (3.9–3.14) and pyparsing versions
(3.1.1 through 3.3.0), aggregating results into
perf_pyparsing.csvat the repo root.tests/README.mdfor usage instructions.Used performance benchmarking to identify and revert an inefficient utility method used in
transform_string(introduced in pyparsing 3.2.0b2).Version 3.2.5 - September, 2025
Wordexpressions that include a spacecharacter, if that expression was then copied, either directly with .copy() or
by adding a results name, or included in another construct (like
DelimitedList)that makes a copy internally. Issue #618, reported by mstinberg, among others -
thanks, and sorry for the inconvenience.
Version 3.2.4 - September, 2025
Barring any catastrophic bugs in this release, this will be the last release in
the 3.2.x line. The next release, 3.3.0, will begin emitting
DeprecationWarningswhen the pre-PEP8 methods are used (see header notes above for more information,
including available automation for converting any existing code using
pyparsing with the old names).
Fixed bug when using a copy of a
Wordexpression (either by using the explicitcopy()method, or attaching a results name), and setting a new expression name,a raised
ParseExceptionstill used the original expression name. Also affectedRegexexpressions withas_matchoras_group_list= True. Reported byWaqas Ilyas, in Issue #612 - good catch!
Fixed type annotation for
replace_with, to acceptAnytype. Fixes Issue #602,reported by esquonk.
Added locking around potential race condition in
ParserElement.reset_cache, aswell as other cache-related methods. Fixes Issue #604, reported by CarlosDescalziIM.
Substantial update to docstrings and doc generation in preparation for 3.3.0,
great effort by FeRD, thanks!
Notable addition by FeRD to convert docstring examples to work with doctest! This
was long overdue, thanks so much!
Version 3.2.3 - March, 2025
nested_exprcould overwrite parse actionsfor defined content, and could truncate list of items within a nested list.
Fixes Issue #600, reported by hoxbro and luisglft, with helpful diag logs and
repro code.
Version 3.2.2 - March, 2025
Released
cvt_pyparsing_pep8_names.pyconversion utility to upgrade pyparsing-basedprograms and libraries that use legacy camelCase names to use the new PEP8-compliant
snake_case method names. The converter can also be imported into other scripts as
Fixed bug in
nested_exprwhere nested contents were stripped of whitespace whenthe default whitespace characters were cleared (raised in this StackOverflow
question https://stackoverflow.com/questions/79327649 by Ben Alan). Also addressed
bug in resolving PEP8 compliant argument name and legacy argument name.
Fixed bug in
rest_of_lineand the underlyingRegexclass, in which matching apattern that could match an empty string (such as
".*"or"[A-Z]*"would not raisea
ParseExceptionat or beyond the end of the input string. This could cause aninfinite parsing loop when parsing
rest_of_lineat the end of the input string.Reported by user Kylotan, thanks! (Issue #593)
Enhancements and extra input validation for
pyparsing.util.make_compressed_re- seeusage in
examples/complex_chemical_formulas.pyand result in the generated railroaddiagram
examples/complex_chemical_formulas_diagram.html. Properly escapes characterslike "." and "*" that have special meaning in regular expressions.
Fixed bug in
one_of()to properly escape characters that are regular expression markers(such as '*', '+', '?', etc.) before building the internal regex.
Better exception message for
MatchFirstandOrexpressions, showing all alternativesrather than just the first one. Fixes Issue #592, reported by Focke, thanks!
Added return type annotation of "-> None" for all
__init__()methods, to satisfymypy --stricttype checking. PR submitted by FeRD, thank you!Added optional argument
show_hiddentocreate_diagramto showelements that are used internally by pyparsing, but are not part of the actual
parser grammar. For instance, the
Tagclass can insert values into the parsedresults but it does not actually parse any input, so by default it is not included
in a railroad diagram. By calling
create_diagramwithshow_hidden = True,these internal elements will be included. (You can see this in the tag_metadata.py
script in the examples directory.)
Fixed bug in
number_words.pyexample. Also addedebnf_number_words.pyto demonstrateusing the
ebnf.pyEBNF parser generator to build a similar parser directly fromEBNF.
Fixed syntax warning raised in
bigquery_view_parser.py, invalid escape sequence "\s".Reported by sameer-google, nice catch! (Issue #598)
Added support for Python 3.14.
Version 3.2.1 - December, 2024
Updated generated railroad diagrams to make non-terminal elements links to their related
sub-diagrams. This greatly improves navigation of the diagram, especially for
large, complex parsers.
Simplified railroad diagrams emitted for parsers using
infix_notation, by hidinglookahead terms. Renamed internally generated expressions for clarity, and improved
diagramming.
Improved performance of
cpp_style_comment,c_style_comment,common.fnumberand
common.ieee_floatRegexexpressions. PRs submitted by Gabriel Gerlero,nice work, thanks!
Add missing type annotations to
match_only_at_col,replace_with,remove_quotes,with_attribute, andwith_class. Issue #585 reported by rafrafrek.Added generated diagrams for many of the examples.
Replaced old
examples/0README.htmlfile withexamples/README.mdfile.Version 3.2.0 - October, 2024
Discontinued support for Python 3.6, 3.7, and 3.8. Adopted new Python features from
Python versions 3.7-3.9:
imported from the
typingmodule (e.g.,list[str]vsList[str]).in dicts (including removal of uses of
OrderedDict).pdb.set_trace()call inParserElement.set_break()tobreakpoint().typing.NamedTupletodataclasses.dataclassin railroad diagrammingcode.
from __future__ import annotationsto clean up some type annotations.(with assistance from ISyncWithFoo, issue #535, thanks for the help!)
POSSIBLE BREAKING CHANGES
The following bugfixes may result in subtle changes in the results returned or
exceptions raised by pyparsing.
Fixed code in
ParseElementEnhancesubclasses thatreplaced detailed exception messages raised in contained expressions with a
less-specific and less-informative generic exception message and location.
If your code has conditional logic based on the message content in raised
ParseExceptions, this bugfix may require changes in your code.Fixed bug in
transform_string()where whitespacein the input string was not properly preserved in the output string.
If your code uses
transform_string, this bugfix may require changes inyour code.
Fixed bug where an
IndexErrorraised in a parse action wasincorrectly handled as an
IndexErrorraised as part of theParserElementparsing methods, and reraised as a
ParseException. Now anIndexErrorthat raises inside a parse action will properly propagate out as an
IndexError.(Issue #573, reported by August Karlstedt, thanks!)
If your code raises
IndexErrors in parse actions, this bugfix may requirechanges in your code.
FIXES AND NEW FEATURES
Added type annotations to remainder of
pyparsingpackage, and addedmypyrun to
tox.ini, so that type annotations are now run as part of pyparsing's CI.Addresses Issue #373, raised by Iwan Aucamp, thanks!
Exception message format can now be customized, by overriding
ParseBaseException.format_message:(PR #571 submitted by Odysseyas Krystalakos, nice work!)
run_testsnow detects if an exception is raised in a parse action, and willreport it with an enhanced error message, with the exception type, string,
and parse action name.
QuotedStringnow handles translation of escaped integer, hex, octal, andUnicode sequences to their corresponding characters.
Fixed the displayed output of
Regexterms to deduplicate repeated backslashes,for easier reading in debugging, printing, and railroad diagrams.
Fixed (or at least reduced) elusive bug when generating railroad diagrams,
where some diagram elements were just empty blocks. Fix submitted by RoDuth,
thanks a ton!
Fixed railroad diagrams that get generated with a parser containing a
Regexelementdefined using a verbose pattern - the pattern gets flattened and comments removed
before creating the corresponding diagram element.
Defined a more performant regular expression used internally by
common_html_entity.Regexinstances can now be created using a callable that takes no argumentsand just returns a string or a compiled regular expression, so that creating complex
regular expression patterns can be deferred until they are actually used for the first
time in the parser.
Added optional
flattenBoolean argument toParseResults.as_list(), toreturn the parsed values in a flattened list.
Added
indentandbase_1arguments topyparsing.testing.with_line_numbers. Whenusing
with_line_numbersinside a parse action, setbase_1=False, since thereported
locvalue is 0-based.indentcan be a leading string (typically ofspaces or tabs) to indent the numbered string passed to
with_line_numbers.Added while working on #557, reported by Bernd Wechner.
NEW/ENHANCED EXAMPLES
Added query syntax to
mongodb_query_expression.pywith:"contains any", and "contains none")
and "=~" operator to support regex matching
a[0]style array referencingAdded
lox_parser.pyexample, a parser for the Lox language used as a tutorial inRobert Nystrom's "Crafting Interpreters" (http://craftinginterpreters.com/).
With helpful corrections from RoDuth.
Added
complex_chemical_formulas.pyexample, to add parsing capability forformulas such as "3(C₆H₅OH)₂".
Updated
tag_emitter.pyto use newTagclass, introduced in pyparsing3.1.3.
Version 3.1.4 - August, 2024
referenced
re.Pattern. Since this type was introduced in Python 3.7, using this typedefinition broke Python 3.6 installs of pyparsing 3.1.3. PR submitted by Felix Fontein,
nice work!
Version 3.1.3 - August, 2024
Added new
TagParserElement, for inserting metadata into the parsed results.This allows a parser to add metadata or annotations to the parsed tokens.
The
Tagelement also accepts an optionalvalueparameter, defaulting toTrue.See the new
tag_metadata.pyexample in theexamplesdirectory.Example:
add tag indicating mood
prints:
Added example
mongodb_query_expression.py, to convert human-readable infix queryexpressions (such as
a==100 and b>=200) and transform them into the equivalentquery argument for the pymongo package (
{'$and': [{'a': 100}, {'b': {'$gte': 200}}]}).Supports many equality and inequality operators - see the docstring for the
transform_queryfunction for more examples.Fixed issue where PEP8 compatibility names for
ParserElementstatic methods werenot themselves defined as
staticmethods. When called using aParserElementinstance,this resulted in a
TypeErrorexception. Reported by eylenburg (#548).To address a compatibility issue in RDFLib, added a property setter for the
ParserElement.nameproperty, to callParserElement.set_name.Modified
ParserElement.set_name()to accept a None value, to clear the definedname and corresponding error message for a
ParserElement.Updated railroad diagram generation for
ZeroOrMoreandOneOrMoreexpressions withstop_onexpressions, while investigating #558, reported by user Gu_f.Added
<META>tag to HTML generated for railroad diagrams to force UTF-8 encodingwith older browsers, to better display Unicode parser characters.
Fixed some cosmetics/bugs in railroad diagrams:
show_groups=Falseshow_results_names=TrueSome type annotations added for parse action related methods, thanks August
Karlstedt (#551).
Added exception type to
trace_parse_actionexception output, while investigatingSO question posted by medihack.
Added
set_namecalls to internal expressions generated ininfix_notation, forimproved railroad diagramming.
delta_time,lua_parser,decaf_parser, androman_numeralsexamples cleaned upto use latest PEP8 names and add minor enhancements.
Fixed bug (and corresponding test code) in
delta_timeexample that did not handleweekday references in time expressions (like "Monday at 4pm") when the weekday was
the same as the current weekday.
Minor performance speedup in
trim_arity, to benefit any parsers using parse actions.Added early testing support for Python 3.13 with JIT enabled.
Version 3.1.2 - March, 2024
Added
ieee_floatexpression topyparsing.common, which parses float values,plus "NaN", "Inf", "Infinity". PR submitted by Bob Peterson (#538).
Updated pep8 synonym wrappers for better type checking compatibility. PR submitted
by Ricardo Coccioli (#507).
Fixed empty error message bug, PR submitted by InSync (#534). This should return
pyparsing's exception messages to a former, more helpful form. If you have code that
parses the exception messages returned by pyparsing, this may require some code
changes.
Added unit tests to test for exception message contents, with enhancement to
pyparsing.testing.assertRaisesParseExceptionto accept an expected exception message.Updated example
select_parser.pyto use PEP8 names and added Groups for better retrievalof parsed values from multiple SELECT clauses.
Added example
email_address_parser.py, as suggested by John Byrd (#539).Added example
directx_x_file_parser.pyto parse DirectX template definitions, andgenerate a Pyparsing parser from a template to parse .x files.
Some code refactoring to reduce code nesting, PRs submitted by InSync.
All internal string expressions using '%' string interpolation and
str.format()converted to f-strings.
Version 3.1.1 - July, 2023
Fixed regression in Word(min), reported by Ricardo Coccioli, good catch! (Issue #502)
Fixed bug in bad exception messages raised by Forward expressions. PR submitted
by Kyle Sunden, thanks for your patience and collaboration on this (#493).
Fixed regression in SkipTo, where ignored expressions were not checked when looking
for the target expression. Reported by catcombo, Issue #500.
Fixed type annotation for enable_packrat, PR submitted by Mike Urbach, thanks
Configuration
📅 Schedule: (in timezone America/New_York)
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by Mend Renovate.