Changing data within bytes... - Printable Version

Changing data within bytes... - Printable Version

+- Southperry.net (https://www.southperry.net)
+-- Forum: Social (https://www.southperry.net/forumdisplay.php?fid=14)
+--- Forum: Rubik's Cube (https://www.southperry.net/forumdisplay.php?fid=58)
+--- Thread: Changing data within bytes... (/showthread.php?tid=13632)

Changing data within bytes... - Nikkey - 2009-07-17

Tends to sometimes be rather hard when it shouldn't be. You may be able to note this especially well when working with bits and strings, where you have to modify characters yourself.

Or if you're a LISPer and is really annoyed by the fact that there's no general function for any sequence (Though there is, elt, but it's slower compared to other specialized functions).

So yeah, I got bored and modified some setf-functions and made a macro character which works like array-refering or hash-refering in python.

Spoiler

Code:
(define-setf-expander logbitp (bit n &environment env)

  (multiple-value-bind (%vals-temps %vals %newvals %setter %getter)

    (get-setf-expansion n env)

    (let ((bit-temp (gensym))

          (newval (gensym))

          (%newval (first %newvals)))

      (values (cons bit-temp %vals-temps)

              (cons bit %vals)

              (list newval)

              `(let ((,%newval (dpb (if (or

                                          (eq ,newval t)

                                          (eq ,newval 1))

                                          1 0)

                                    (byte 1 ,bit-temp)

                                    ,%getter)))

                 ,%setter ; The %setter form will do the dirty work if %newval is bound.

                 ,newval)

              `(logbitp ,bit-temp ,%vals-temps)))))

(define-setf-expander char-int (char &environment env)

  (multiple-value-bind (%vals-temps %vals %newvals %setter %getter)

      (get-setf-expansion char env)

    (let ((newval (gensym)))

      (values %vals-temps

              %vals

              (list newval)

              `(progn (setf ,%getter (int-char ,newval)) ,newval)

              `(char-int ,%getter)))))

(set-macro-character #\] (get-macro-character #\)))

(set-dispatch-macro-character #\# #\[

  #'(lambda (stream char1 char2)

      (let ((lst (read-delimited-list #\] stream t)))

        (labels ((make-expression (expr points)

                   (if points

                      (typecase (eval expr)

                        (list   (make-expression `(nth ,(car points) ,expr) (cdr points)))

                        (string (make-expression `(schar ,expr ,(car points)) (cdr points)))

                        (vector (make-expression `(svref ,expr ,(car points)) (cdr points)))

                        (hash-table (make-expression `(gethash ,(car points) ,expr) (cdr points)))

                        (array `(aref ,expr ,@points))

                        (integer `(logbitp ,@points ,expr))

                        (character `(logbitp ,@points (char-int ,expr)))

                        (t (make-expression `(elt ,expr ,(car points)) (cdr points))))

                      expr)))

          (make-expression (car lst) (cdr lst))))))

Simply saying, one can refer to any sequence or array, including characters and integers by this #[]-macro:

Code:
> #[3 1] ; Second bit in the integer 3

T

> #[#\d 0] ; First bit in the character d

NIL

> (setf x #\d)

> (setf #[x 0] 1) ; Changing first bit from 0 to 1

1

> x ; x is now modified

#\e

> (setf x (make-hash-table)

        #[x 55] #("A vector" "consisting of" "strings")))

> #[x 55]

#("A vector" "consisting of" "strings")

> #[x 55 2]

"strings"

> #[x 55 2 2]

#\r

> #[x 55 2 2 1] ; Equals to (logbitp (char-int (schar (svref (gethash 55 x) 2) 2)) 1), or the second bit in #\r

T

> (setf #[x 55 2 2 1] nil)

NIL

> #[x 55 2]

"stpings"

Can't make things like this in C, python, perl or anything else, eh? Tongue1

I'm working on making it able to make dispatched arrays and such, so you can modify strings, arrays and whatever else destructively. Kinda like Python's [1:2:3] way of dealing with arrays, just through '(1 2 3) instead.

A destructive string-xoring in lisp, through one change in the #[-macro and adding in a (setf logxor)-function would be like this:

Code:
(defun string-xor (string1 string2)

  (let ((lst (coerce string2 'list)))

    (loop with circular-lst = (nconc lst lst) ; Circular list

          for i below (length string1)

          do (setf (logxor #[string1 i]) #[circular-lst i]))))

; And here's a non-destructive one:

(defun non-destructive-string-xor (string1 string2) ; Might want to shorten the name of this one

  (let ((lst (coerce string2 'list)))

    (with-output-to-string (out)

      (loop with circular-lst = (nconc lst lst)

            for i below (length string1)

            do (write-character (int-char (logxor #[string1 i] #[circular-lst i]))

                 out)))))

Which looks way prettier and easier to understand than your C or python string-xoring Stunned

Spoiler

Changing data within bytes... - Fiel - 2009-07-17

Python does not allow strings or chars to be xor'd. You have to change them from strings to bytes, xor them, then convert back.

Code:
>>> MyChar = "P"

>>> MyChar ^ 0xFF

Traceback (most recent call last):

  File "<pyshell#1>", line 1, in <module>

    MyChar ^ 0xFF

TypeError: unsupported operand type(s) for ^: 'str' and 'int'

>>> import struct

>>> MyChar = struct.unpack('b', MyChar)[0] #Essentially atoi(). 'b' means "signed byte"

>>> print(MyChar)

80

>>> MyChar ^ 0xFF

175

Changing data within bytes... - Nikkey - 2009-07-17

Fiel Wrote:Python does not allow strings or chars to be xor'd. You have to change them from strings to bytes, xor them, then convert back.

Code:
>>> MyChar = "P" >>> MyChar ^ 0xFF Traceback (most recent call last): File "<pyshell#1>", line 1, in <module> MyChar ^ 0xFF TypeError: unsupported operand type(s) for ^: 'str' and 'int' >>> import struct >>> MyChar = struct.unpack('b', MyChar)[0] #Essentially atoi(). 'b' means "signed byte" >>> print(MyChar) 80 >>> MyChar ^ 0xFF 175

I don't know any language which allows the simple
str1 ^ str2-feat, nor the char ^ int, but in python it would be something like

Code:
import struct

def xor(str1, str2):

  str1Lst = struct.unpack("%db" % len(str1), str1)

  str2Lst = struct.unpack("%db" % len(str2), str2)

  return struct.pack("%db" % len(str1), map (lambda x, y: x^y, str1Lst, str2Lst))

Now, that would require the same size on str1 and str2, but repeating str2 would be rather easy to produce, I'd assume.

I could eventually modify logxor in lisp so it would work as the non-destructive-string-xor-function, but it would require stuff I've not worked with as of right now. Besides, there's really no need for it in Lisp, unless you send in data to the xor-function which you don't know whether's a number or a string. But then again, you could just check its type.

Changing data within bytes... - AngelSL - 2009-07-17

Umm.. what?!

Well I don't do LISP or Python so..

Changing data within bytes... - Nikkey - 2009-07-17

AngelSL Wrote:Umm.. what?!

Well I don't do LISP or Python so..

In lisp you can make your own macros and symbol-macros, which can ease your work by a lot. For example, lisp does not have for-loops or while-loops (and interestingly enough, I've never had the need to make a for-macro either). You can, however, make a while loop by simply writing this:

Code:
(defmacro while (test &body body)

  `(do ()

       ((not ,test))

     ,@body))

(This is the easiest macro I know of really, I don't know if I'm able to make an easier.)

The lisp macros I defined above makes me able to avoid long expressions which regards arrays, bits, characters and hash-tables, and focus on the main work instead. For example, if I have a hash-table, which has a key which points towards an array, and that array is filled with strings, and I want to either change a character or a string, I will be able to do so easily through this macro (The expression would be equal to #[ht key strnum charnum], instead of the long and big (char (aref (gethash key ht) strnum) charnum).)

Changing data within bytes... - Spaz - 2009-07-17

Lisp: where you have to modify the language to get anything useful done.

I don't really get what's going on here. You made a macro/function that lets you take the nth element of any sequence? Doesn't sound that special to me. :-\

Changing bits isn't hard if you write a function for it. In C++:

Spoiler

Edit: I wrote that code a while ago. A non-template function taking a void* would result in smaller code.

Changing data within bytes... - Dusk - 2009-07-17

Devil's Sunrise Wrote:Can't make things like this in C, python, perl or anything else, eh?

I think you can in C *scratches head* In py/pl you can do something to the same purpose without writing a whole crapload of code. You wouldn't usually use Perl for the same things you'd want Lisp for anyway.

whee

Changing data within bytes... - Kortestanov - 2009-07-17

If you just want a quick and simple way to xor every character in a string without doing too many convertions, just make a dll in asm and make an external dll call. This way you will also gain some performance, if that matters to you.

Changing data within bytes... - Nikkey - 2009-07-17

Spaz Wrote:Lisp: where you have to modify the language to get anything useful done.

I don't really get what's going on here. You made a macro/function that lets you take the nth element of any sequence? Doesn't sound that special to me. :-\

Changing bits isn't hard if you write a function for it. In C++:
Spoiler

Code:
template<typename type> void setbit(type& variable, size_t bit, bool on) { unsigned char * bytePtr = (unsigned char *) (&variable); size_t byteOffset = bit / 8; bytePtr += byteOffset; size_t bitOffset = bit % 8; if(on) { *bytePtr = *bytePtr | ( (unsigned char) (1) << (7 - bitOffset)); } else { *bytePtr = *bytePtr & ( ( ~((unsigned char) (1)) ) << (7 - bitOffset)); } } template<typename type> bool getbit(const type& variable, size_t bit) { const unsigned char * bytePtr = (const unsigned char *) (&variable); size_t byteOffset = bit / 8; bytePtr += byteOffset; size_t bitOffset = bit % 8; unsigned char byteCopy = *bytePtr; byteCopy = byteCopy | ( (unsigned char) (1) << (7 - bitOffset)); // Set the bit in a copy of the byte to 1, then check if the setting of the bit changed the byte if(byteCopy == *bytePtr) { return true; } else { return false; } }

Thing is, it's easier to refer to the bit through this, correct?

Code:
(un)signed whatever n = x;

int bitnumber = y;

cout << x[bitnumber];

x[bitnumber] = 1;

Dusk Wrote:I think you can in C *scratches head* In py/pl you can do something to the same purpose without writing a whole crapload of code. You wouldn't usually use Perl for the same things you'd want Lisp for anyway.

whee

print ord('P') ^ 0xFF;

Oh, that's really fancy. I don't know what you'd use Perl especially for, though I know you guys have a Memoization module:

Spoiler

If you're able to write modules which work like this, then it's amazing. In LISP, you could always make a macro which do the same thing:

Spoiler

Though this would be rather slow for some cases (like where you can use an n-dimensional array instead), and possibly more inefficient unless you optimise the code some.

Kortestanov Wrote:If you just want a quick and simple way to xor every character in a string without doing too many convertions, just make a dll in asm and make an external dll call. This way you will also gain some performance, if that matters to you.

I suppose you could, and I suppose it wouldn't be too hard/ugly to implement something like that. But that's not really the idea behind this function. The idea was just to show lisp's flexibility, and hopefully something we would be able to do later on. I'm aware that changing bits (and bytes) in programming-languages doesn't occur extremely frequently, but since there's nothing to lose by having this function available, then why not implement it to ease the work for programmers?

Changing data within bytes... - Spaz - 2009-07-17

Devil's Sunrise Wrote:Thing is, it's easier to refer to the bit through this, correct?

Code:
(un)signed whatever n = x; int bitnumber = y; cout << x[bitnumber]; x[bitnumber] = 1;

No, it's not any easier than the below code. In fact, I think the code below is more explicit about what it's doing and thus easier to read.

Code:
(un)signed whatever n = x;

int bitnumber = y;

cout << getbit(n, bitnmber);

setbit(n, bitnumber, true);

Changing data within bytes... - Fiel - 2009-07-17

Do these programming languages also work on big endian processors? That could make all of this wrong.

Changing data within bytes... - Nikkey - 2009-07-17

Spaz Wrote:No, it's not any easier than the below code. In fact, I think the code below is more explicit about what it's doing and thus easier to read.

Code:
(un)signed whatever n = x; int bitnumber = y; cout << getbit(n, bitnmber); setbit(n, bitnumber, true);

Well, I suppose that's based on preference. For example, a Java/C/C++ programmer would most likely scream in pain if he's told to actually write the following:

Code:
int array = makeArray(x);

int pos = y;

cout << get(array, pos);

set(array, pos, z);

When the standardized way is to write the following:

Code:
int array[x];

int pos = y;

cout << array[pos];

array[pos] = z;

But you can't deny that referring to n[y] is shorter than getbit(n, y), and that referring to n[y] when defined within a compiler could be so efficient that it would point directly to that bit in the memory. (getbit and setbit, if defined within a compiler, could do the same thing of course)

Fiel Wrote:Do these programming languages also work on big endian processors? That could make all of this wrong.

I've never thought about that issue, really. That should however be the compiler's, and not the programmers' issue. A piece of uncompiled code shouldn't have to be modified in order to work on another computing system. That should the compilers take care of.