Skip to content

Remove torchtext #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jul 30, 2025
Merged

Remove torchtext #22

merged 9 commits into from
Jul 30, 2025

Conversation

Eleven1Liu
Copy link
Contributor

@Eleven1Liu Eleven1Liu commented Jul 22, 2025

What does this PR do?

  • Remove torchtext and update related functions (mostly in nn/data_utils.py)

  • Dependencies: Remove versions for pytorch (<=2.3.0 for torchtext) and transformers (see PR Upgrade minimum python version from 3.8 to 3.10 #20).

    • Package versions with and without torchtext
    torch CUDA transformers
    with torchtext <=2.3.0 11.8/12.1 <=4.51.3
    w/o torchtext (this PR) 2.3.0~2.7.1 11.8/12.1/12.6 4.54.0 (tested version)
    • Test results
      ❌ : ERROR: Could not find a version that satisfies the requirement torch==2.x.x (from versions: 2.5.0, 2.5.1, 2.6.0, 2.7.0, 2.7.1), ERROR: No matching distribution found for torch==2.x.x

      cuda pytorch python 3.10.18 python 3.13.5
      11.8 2.3.0
      11.8 2.3.1
      11.8 2.4.1
      11.8 2.5.1
      11.8 2.6.0
      11.8 2.7.1
      12.1 2.3.0
      12.1 2.3.1
      12.1 2.4.1
      12.1 2.5.1
      12.6 2.6.0
      12.6 2.7.1
    • torch < 2.3.0 is incompatible with numpy 2.x (note). The error message is as follows:

    A module that was compiled using NumPy 1.x cannot be run in 
    NumPy 2.2.6 as it may crash. To support both 1.x and 2.x
    versions of NumPy, modules must be compiled with NumPy 2.0. 
    Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.
    
    If you are a user of the module, the easiest solution will be to
    downgrade to 'numpy<2' or try to upgrade the affected module.
    We expect that some modules will need time to support NumPy 2.
    

Test CLI & API (bash tests/autotest.sh)

Test APIs used by main.py.

  • Test Pass
    • (Copy and paste the last outputted line here.)
  • Not Applicable (i.e., the PR does not include API changes.)

Check API Document

If any new APIs are added, please check if the description of the APIs is added to API document.

  • API document is updated (linear, nn)
  • Not Applicable (i.e., the PR does not include API changes.)

Test quickstart & API (bash tests/docs/test_changed_document.sh)

If any APIs in quickstarts or tutorials are modified, please run this test to check if the current examples can run correctly after the modified APIs are released.

@Eleven1Liu Eleven1Liu linked an issue Jul 22, 2025 that may be closed by this pull request
@Eleven1Liu Eleven1Liu added enhancement New feature or request model/nn labels Jul 22, 2025
@Eleven1Liu Eleven1Liu marked this pull request as ready for review July 27, 2025 09:33
@Eleven1Liu Eleven1Liu requested review from cjlin1 and a team as code owners July 27, 2025 09:33
@Eleven1Liu Eleven1Liu added the package versions Versions for related packages label Jul 27, 2025
@Eleven1Liu Eleven1Liu self-assigned this Jul 29, 2025
@Eleven1Liu Eleven1Liu changed the title [WIP] Remove torchtext Remove torchtext Jul 29, 2025
@Eleven1Liu Eleven1Liu requested a review from maclin726 July 29, 2025 11:33
@Eleven1Liu Eleven1Liu merged commit ac49f51 into ntumlgroup:master Jul 30, 2025
1 check passed
@Eleven1Liu Eleven1Liu deleted the free_torchtext branch July 30, 2025 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request model/nn package versions Versions for related packages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove torchtext to unfreeze torch
2 participants