Increase maximum characters in read-line

[snozawa]

I'd like to read lines which has more than 8192 characters .
Currently, we can read lines until 8192 characters because of immediate in lisp/c/lispio.c.

Is there any reason to use 8192?
Any problems with the following increasing?

Index: lispio.c
===================================================================
--- lispio.c    (リビジョン 669)
+++ lispio.c    (作業コピー)
@@ -212,13 +212,13 @@
 pointer argv[];
 { register pointer strm;
   pointer eoferrorp=T,eofvalue=NIL;
-  byte cb[8192];
+  byte cb[16384];
   register int i=0,ch;
   ckarg2(0,3);
   strm=getinstream(ctx,n,argv[0]);
   if (n>=2) eoferrorp=argv[1];
   if (n==3) eofvalue=argv[2];
-  while (i<8192) {
+  while (i<16384) {
     ch=readch(strm);
     if (ch=='\n') break;
     else if (ch==EOF) {

[garaemon]

You can use triple backquotes when pasting codes. It will have pretty highlighting.

[snozawa]

Thanks, I fixed it.

[garaemon]

anyway,

8192 bytes ~= 8k bytes
16384 bytes ~= 16k bytes

I think 16K Bytes OK.

In Java case, it uses 8k bytes buffer as default.
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/io/BufferedReader.java#BufferedReader

[k-okada]

you can use ｀｀｀ for no formatting text (verbatim in tex)

｀｀｀
プログラム
｀｀｀

On Thursday, March 6, 2014, snozawa notifications@github.com wrote:

I'd like to read lines which has more than 8192 characters .
Currently, we can read lines until 8192 characters because of immediate in
lisp/c/lispio.c.

Is there any reason to use 8192?
Any problems with the following increasing?
Index: lispio.c

--- lispio.c (リビジョン 669)
+++ lispio.c (作業コピー)
@@ -212,13 +212,13 @@
pointer argv[];
{ register pointer strm;
pointer eoferrorp=T,eofvalue=NIL;

• byte cb[8192];
• byte cb[16384]; register int i=0,ch; ckarg2(0,3);
  strm=getinstream(ctx,n,argv[0]); if (n>=2) eoferrorp=argv[1]; if (n==3)
  eofvalue=argv[2];
• while (i<8192) {
• while (i<16384) { ch=readch(strm); if (ch=='\n') break; else if
  (ch==EOF) {

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2
.

[YoheiKakiuchi]

Is using fixed length OK?
Anyway, error or warning message should be added if line length reach limitation.

[k-okada]

Is using fixed length OK?

that's good point. even if we exted to 16K, we will hit the limit. what is your use case? which lisp function are you using? maybe we can also fit the problem at that point.

we need separate pull request

• error or warning message should be added if line length reach limitation.
• proposal for extend the limitation

[snozawa]

that's good point. even if we exted to 16K, we will hit the limit.

Yes, it would be better to set limitation.

what is your use case? which lisp function are you using? maybe we can also fit the problem at that point.

In my case, I'd like to read string from piped-fork.
In this case, the string exceeded 8192 limitation.

error or warning message should be added if line length reach limitation.
proposal for extend the limitation

OK, I'll try both of them.

[snozawa]

error or warning message should be added if line length reach limitation.

I sent PReq to add a line to throw error if maximum character exceeded.

proposal for extend the limitation

For this purpose, I'll write a PReq to configure maximum character
by using global variable such as:

 *maximum-read-line-character*

[k-okada]

i think we should write something like this

pointr read_line() {
 start_over:
  while (...) {
  }
  ret = concatenate(ret, make_string())
  if NOT_FINISHED:
   goto start_over:
 return ret;
}

, or just leave 8192 or other number

having global variable meas we solved problem but it (if you appropriately choose hidden magic number).

[garaemon]

@k-okada 's proposal seems good.

This is the same approach used in CPython implementation. (actually it's not the same 😄)

    def readline(self, limit=-1):
        r"""Read and return a line from the stream.

        If limit is specified, at most limit bytes will be read.

        The line terminator is always b'\n' for binary files; for text
        files, the newlines argument to open can be used to select the line
        terminator(s) recognized.
        """
        # For backwards compatibility, a (slowish) readline().
        if hasattr(self, "peek"):
            def nreadahead():
                readahead = self.peek(1)
                if not readahead:
                    return 1
                n = (readahead.find(b"\n") + 1) or len(readahead)
                if limit >= 0:
                    n = min(n, limit)
                return n
        else:
            def nreadahead():
                return 1
        if limit is None:
            limit = -1
        elif not isinstance(limit, (int, long)):
            raise TypeError("limit must be an integer")
        res = bytearray()
        while limit < 0 or len(res) < limit:
            b = self.read(nreadahead())
            if not b:
                break
            res += b
            if res.endswith(b"\n"):
                break
        return bytes(res)