How to unescape a Java string literal in Java?
I'm processing some Java source code using Java. I'm extracting the string literals and feeding them to a function taking a String. The problem is that I need to pass the unescaped version of the String to the function (ie. this means converting n to a newline, and to a single , etc).
Is there a function inside the Java API that does this? If not, can I obtain such functionality from some library? Obviously the Java compiler has to do this conversion.
PS - In case anyone wants to know: I'm trying to unobfuscate string literals in decompiled obfuscated Java files
EDIT: You can download the full source for the function I discuss below. I also discuss it in more detail in this answer.
The Problem
The org.apache.commons.lang.StringEscapeUtils.unescapeJava() given here as “the answer” is really very little help at all.
for null. java.util.regex.Pattern.compile() and everything that uses it, including a , e , and especially cX . writer argument, so if you don't pass it one it still has to create a dummy StringWriter for the output, then convert that to pass back to you. charAt interface instead of the codePoint interface, thus promulgating the delusion that a Java char is guaranteed to hold a Unicode character. It's not. They only get away with this blindness to the astral planes because no UTF-16 surrogate will wind up looking for anything they're looking for. Like many of the other points, their embarrassing ignorance about the names of code points U+2F and U+5C instills no confidence in them whatsoever. For the record:
/ 47 002F SOLIDUS
= slash, virgule
x (latin letter dental click - 01C0)
x (combining long solidus overlay - 0338)
x (fraction slash - 2044)
x (division slash - 2215)
92 005C REVERSE SOLIDUS
= backslash
x (combining reverse solidus overlay - 20E5)
x (set minus - 2216)
The Solution
So this morning I finally got fed up with not being able to read in strings with embedded escapes in them. I needed it for writing the test suite for a larger and more intersting project: transparently converting Java's indefensibly Unicode-ignorant regular expressions into versions where you can use all of w , W , s , S , v , V , h , H , d , D , b , B , X , and R in your patterns and have them actually work properly with Unicode. All I do is rewrite the pattern string; it still compiles with the standard java.util.regex.Pattern.compile() function, so everything works as expected. The string unescaper intentionally passes any b 's through untouched, in case you call it before you call the converter function to make Java regexes Unicode-aware, since that has to deal with b in the boundary sense.
Anyway, here's the string unescaper, which although the less interesting of the pair, does solve the OP's question without all the irritations of the Apache code. It could handle a bit of tightening in a couple places, but I quickly hacked it out over a few hours before lunch just to get it up and running to help drive the test suite. The other function is a lot more work: that one took me all day yesterday, darn it.
/*
*
* unescape_perl_string()
*
* Tom Christiansen <tchrist@perl.com>
* Sun Nov 28 12:55:24 MST 2010
*
* It's completely ridiculous that there's no standard
* unescape_java_string function. Since I have to do the
* damn thing myself, I might as well make it halfway useful
* by supporting things Java was too stupid to consider in
* strings:
*
* => "?" items are additions to Java string escapes
* but normal in Java regexes
*
* => "!" items are also additions to Java regex escapes
*
* Standard singletons: ?a ?e f n r t
*
* NB: b is unsupported as backspace so it can pass-through
* to the regex translator untouched; I refuse to make anyone
* doublebackslash it as doublebackslashing is a Java idiocy
* I desperately wish would die out. There are plenty of
* other ways to write it:
*
* cH, 12, 12, x08 x{8}, u0008, U00000008
*
* Octal escapes: N NN N NN NNN
* Can range up to !777 not 377
*
* TODO: add !o{NNNNN}
* last Unicode is 4177777
* maxint is 37777777777
*
* Control chars: ?cX
* Means: ord(X) ^ ord('@')
*
* Old hex escapes: xXX
* unbraced must be 2 xdigits
*
* Perl hex escapes: !x{XXX} braced may be 1-8 xdigits
* NB: proper Unicode never needs more than 6, as highest
* valid codepoint is 0x10FFFF, not maxint 0xFFFFFFFF
*
* Lame Java escape: [IDIOT JAVA PREPROCESSOR]uXXXX must be
* exactly 4 xdigits;
*
* I can't write XXXX in this comment where it belongs
* because the damned Java Preprocessor can't mind its
* own business. Idiots!
*
* Lame Python escape: !UXXXXXXXX must be exactly 8 xdigits
*
* TODO: Perl translation escapes: Q U L E [IDIOT JAVA PREPROCESSOR]u l
* These are not so important to cover if you're passing the
* result to Pattern.compile(), since it handles them for you
* further downstream. Hm, what about [IDIOT JAVA PREPROCESSOR]u?
*
*/
public final static
String unescape_perl_string(String oldstr) {
/*
* In contrast to fixing Java's broken regex charclasses,
* this one need be no bigger, as unescaping shrinks the string
* here, where in the other one, it grows it.
*/
StringBuffer newstr = new StringBuffer(oldstr.length());
boolean saw_backslash = false;
for (int i = 0; i < oldstr.length(); i++) {
int cp = oldstr.codePointAt(i);
if (oldstr.codePointAt(i) > Character.MAX_VALUE) {
i++; /****WE HATES UTF-16! WE HATES IT FOREVERSES!!!****/
}
if (!saw_backslash) {
if (cp == '') {
saw_backslash = true;
} else {
newstr.append(Character.toChars(cp));
}
continue; /* switch */
}
if (cp == '') {
saw_backslash = false;
newstr.append('');
newstr.append('');
continue; /* switch */
}
switch (cp) {
case 'r': newstr.append('r');
break; /* switch */
case 'n': newstr.append('n');
break; /* switch */
case 'f': newstr.append('f');
break; /* switch */
/* PASS a b THROUGH!! */
case 'b': newstr.append("b");
break; /* switch */
case 't': newstr.append('t');
break; /* switch */
case 'a': newstr.append(' 07');
break; /* switch */
case 'e': newstr.append(' 33');
break; /* switch */
/*
* A "control" character is what you get when you xor its
* codepoint with '@'==64. This only makes sense for ASCII,
* and may not yield a "control" character after all.
*
* Strange but true: "c{" is ";", "c}" is "=", etc.
*/
case 'c': {
if (++i == oldstr.length()) { die("trailing c"); }
cp = oldstr.codePointAt(i);
/*
* don't need to grok surrogates, as next line blows them up
*/
if (cp > 0x7f) { die("expected ASCII after c"); }
newstr.append(Character.toChars(cp ^ 64));
break; /* switch */
}
case '8':
case '9': die("illegal octal digit");
/* NOTREACHED */
/*
* may be 0 to 2 octal digits following this one
* so back up one for fallthrough to next case;
* unread this digit and fall through to next case.
*/
case '1':
case '2':
case '3':
case '4':
case '5':
case '6':
case '7': --i;
/* FALLTHROUGH */
/*
* Can have 0, 1, or 2 octal digits following a 0
* this permits larger values than octal 377, up to
* octal 777.
*/
case '0': {
if (i+1 == oldstr.length()) {
/* found at end of string */
newstr.append(Character.toChars(0));
break; /* switch */
}
i++;
int digits = 0;
int j;
for (j = 0; j <= 2; j++) {
if (i+j == oldstr.length()) {
break; /* for */
}
/* safe because will unread surrogate */
int ch = oldstr.charAt(i+j);
if (ch < '0' || ch > '7') {
break; /* for */
}
digits++;
}
if (digits == 0) {
--i;
newstr.append(' ');
break; /* switch */
}
int value = 0;
try {
value = Integer.parseInt(
oldstr.substring(i, i+digits), 8);
} catch (NumberFormatException nfe) {
die("invalid octal value for escape");
}
newstr.append(Character.toChars(value));
i += digits-1;
break; /* switch */
} /* end case '0' */
case 'x': {
if (i+2 > oldstr.length()) {
die("string too short for x escape");
}
i++;
boolean saw_brace = false;
if (oldstr.charAt(i) == '{') {
/* ^^^^^^ ok to ignore surrogates here */
i++;
saw_brace = true;
}
int j;
for (j = 0; j < 8; j++) {
if (!saw_brace && j == 2) {
break; /* for */
}
/*
* ASCII test also catches surrogates
*/
int ch = oldstr.charAt(i+j);
if (ch > 127) {
die("illegal non-ASCII hex digit in x escape");
}
if (saw_brace && ch == '}') { break; /* for */ }
if (! ( (ch >= '0' && ch <= '9')
||
(ch >= 'a' && ch <= 'f')
||
(ch >= 'A' && ch <= 'F')
)
)
{
die(String.format(
"illegal hex digit #%d '%c' in x", ch, ch));
}
}
if (j == 0) { die("empty braces in x{} escape"); }
int value = 0;
try {
value = Integer.parseInt(oldstr.substring(i, i+j), 16);
} catch (NumberFormatException nfe) {
die("invalid hex value for x escape");
}
newstr.append(Character.toChars(value));
if (saw_brace) { j++; }
i += j-1;
break; /* switch */
}
case 'u': {
if (i+4 > oldstr.length()) {
die("string too short for u escape");
}
i++;
int j;
for (j = 0; j < 4; j++) {
/* this also handles the surrogate issue */
if (oldstr.charAt(i+j) > 127) {
die("illegal non-ASCII hex digit in u escape");
}
}
int value = 0;
try {
value = Integer.parseInt( oldstr.substring(i, i+j), 16);
} catch (NumberFormatException nfe) {
die("invalid hex value for u escape");
}
newstr.append(Character.toChars(value));
i += j-1;
break; /* switch */
}
case 'U': {
if (i+8 > oldstr.length()) {
die("string too short for U escape");
}
i++;
int j;
for (j = 0; j < 8; j++) {
/* this also handles the surrogate issue */
if (oldstr.charAt(i+j) > 127) {
die("illegal non-ASCII hex digit in U escape");
}
}
int value = 0;
try {
value = Integer.parseInt(oldstr.substring(i, i+j), 16);
} catch (NumberFormatException nfe) {
die("invalid hex value for U escape");
}
newstr.append(Character.toChars(value));
i += j-1;
break; /* switch */
}
default: newstr.append('');
newstr.append(Character.toChars(cp));
/*
* say(String.format(
* "DEFAULT unrecognized escape %c passed through",
* cp));
*/
break; /* switch */
}
saw_backslash = false;
}
/* weird to leave one at the end */
if (saw_backslash) {
newstr.append('');
}
return newstr.toString();
}
/*
* Return a string "U+XX.XXX.XXXX" etc, where each XX set is the
* xdigits of the logical Unicode code point. No bloody brain-damaged
* UTF-16 surrogate crap, just true logical characters.
*/
public final static
String uniplus(String s) {
if (s.length() == 0) {
return "";
}
/* This is just the minimum; sb will grow as needed. */
StringBuffer sb = new StringBuffer(2 + 3 * s.length());
sb.append("U+");
for (int i = 0; i < s.length(); i++) {
sb.append(String.format("%X", s.codePointAt(i)));
if (s.codePointAt(i) > Character.MAX_VALUE) {
i++; /****WE HATES UTF-16! WE HATES IT FOREVERSES!!!****/
}
if (i+1 < s.length()) {
sb.append(".");
}
}
return sb.toString();
}
private static final
void die(String foa) {
throw new IllegalArgumentException(foa);
}
private static final
void say(String what) {
System.out.println(what);
}
As anybody can plainly see from the Java code above, I'm really a C programmer — Java is anything but my favorite language. I'm afraid that I really do have to side with Rob Pike in his famous public static void talk on this one.
'Nuff said.
Anyway, it's only a quick morning's hackery, but if it helps others, you're welcome to it — no strings attached. If you improve it, I'd love for you to mail me your enhancements, but you certainly don't have to.
You can use String unescapeJava(String) method of StringEscapeUtils from Apache Commons Lang.
Here's an example snippet:
String in = "atbn"c"";
System.out.println(in);
// atbn"c"
String out = StringEscapeUtils.unescapeJava(in);
System.out.println(out);
// a b
// "c"
The utility class has methods to escapes and unescape strings for Java, Java Script, HTML, XML, and SQL. It also has overloads that writes directly to a java.io.Writer .
Caveats
It looks like StringEscapeUtils handles Unicode escapes with one u , but not octal escapes, or Unicode escapes with extraneous u s.
/* Unicode escape test #1: PASS */
System.out.println(
"u0030"
); // 0
System.out.println(
StringEscapeUtils.unescapeJava("u0030")
); // 0
System.out.println(
"u0030".equals(StringEscapeUtils.unescapeJava("u0030"))
); // true
/* Octal escape test: FAIL */
System.out.println(
"45"
); // %
System.out.println(
StringEscapeUtils.unescapeJava("45")
); // 45
System.out.println(
"45".equals(StringEscapeUtils.unescapeJava("45"))
); // false
/* Unicode escape test #2: FAIL */
System.out.println(
"uu0030"
); // 0
System.out.println(
StringEscapeUtils.unescapeJava("uu0030")
); // throws NestableRuntimeException:
// Unable to parse unicode value: u003
A quote from the JLS:
Octal escapes are provided for compatibility with C, but can express only Unicode values u0000 through u00FF , so Unicode escapes are usually preferred.
If your string can contain octal escapes, you may want to convert them to Unicode escapes first, or use another approach.
The extraneous u is also documented as follows:
The Java programming language specifies a standard way of transforming a program written in Unicode into ASCII that changes a program into a form that can be processed by ASCII-based tools. The transformation involves converting any Unicode escapes in the source text of the program to ASCII by adding an extra u -for example, uxxxx becomes uuxxxx -while simultaneously converting non-ASCII characters in the source text to Unicode escapes containing a single u each.
This transformed version is equally acceptable to a compiler for the Java programming language and represents the exact same program. The exact Unicode source can later be restored from this ASCII form by converting each escape sequence where multiple u 's are present to a sequence of Unicode characters with one fewer u , while simultaneously converting each escape sequence with a single u to the corresponding single Unicode character.
If your string can contain Unicode escapes with extraneous u , then you may also need to preprocess this before using StringEscapeUtils .
Alternatively you can try to write your own Java string literal unescaper from scratch, making sure to follow the exact JLS specifications.
References
Came across a similar problem, wasn't also satisfied with the presented solutions and implemented this one myself.
Also available as a Gist on Github:
/**
* Unescapes a string that contains standard Java escape sequences.
* <ul>
* <li><strong>\b \f \n \r \t \" \'</strong> :
* BS, FF, NL, CR, TAB, double and single quote.</li>
* <li><strong>\X \XX \XXX</strong> : Octal character
* specification (0 - 377, 0x00 - 0xFF).</li>
* <li><strong>\uXXXX</strong> : Hexadecimal based Unicode character.</li>
* </ul>
*
* @param st
* A string optionally containing standard java escape sequences.
* @return The translated string.
*/
public String unescapeJavaString(String st) {
StringBuilder sb = new StringBuilder(st.length());
for (int i = 0; i < st.length(); i++) {
char ch = st.charAt(i);
if (ch == '') {
char nextChar = (i == st.length() - 1) ? '' : st
.charAt(i + 1);
// Octal escape?
if (nextChar >= '0' && nextChar <= '7') {
String code = "" + nextChar;
i++;
if ((i < st.length() - 1) && st.charAt(i + 1) >= '0'
&& st.charAt(i + 1) <= '7') {
code += st.charAt(i + 1);
i++;
if ((i < st.length() - 1) && st.charAt(i + 1) >= '0'
&& st.charAt(i + 1) <= '7') {
code += st.charAt(i + 1);
i++;
}
}
sb.append((char) Integer.parseInt(code, 8));
continue;
}
switch (nextChar) {
case '':
ch = '';
break;
case 'b':
ch = 'b';
break;
case 'f':
ch = 'f';
break;
case 'n':
ch = 'n';
break;
case 'r':
ch = 'r';
break;
case 't':
ch = 't';
break;
case '"':
ch = '"';
break;
case ''':
ch = ''';
break;
// Hex Unicode: u????
case 'u':
if (i >= st.length() - 5) {
ch = 'u';
break;
}
int code = Integer.parseInt(
"" + st.charAt(i + 2) + st.charAt(i + 3)
+ st.charAt(i + 4) + st.charAt(i + 5), 16);
sb.append(Character.toChars(code));
i += 5;
continue;
}
i++;
}
sb.append(ch);
}
return sb.toString();
}
上一篇: JSONObject.toString：如何不转义斜杠
下一篇: 如何在Java中使用Java字符串文字？