2/26/2013
Hide php version and apache version
Most of hacker may find the bug based on apache version and hack the server.
So you had better hide the apache version.
By default, the apache will send apache version to client in every http header.
Another, the php will send the php version to client.
Now I give the way how to hide the apache version and php version
1. Hide apache version
ServerTokens controls whether Server response header field, which is sent back to clients, includes a descriptions of the generic OS-type of the server.
open httpd.conf and change
ServerSignature On => ServerSignature Off
ServerTokens os=>ProductOnly
2. Hide php version
expose_php = On =>expose_php = Off
2/22/2013
MySQL index performance and example
The aims for index in MySQL is to find row quickly. For example, there is a table like
CREATE TABLE test (
id INT NOT NULL,
last_name CHAR(30) NOT NULL,
first_name CHAR(30) NOT NULL,
PRIMARY KEY (id),
);
If we execute
select * from test where last_name = 'Obama';
the MySQL will check all records in table test. It will be the O(n) questions.
If we use index for this table, it is not necessary to check each row from 0 to n. How to improve the performance to search one or more special rows in table?
For example, when we want to find a keyword in a book, we find it in the index list and then get the page no from it. We can find it quickly based on the book index. MySQL may also use this method to get it by index.
Now we use command
alter table test add index(last_name);
We have added one index on table test for last_name. if we execute
select * from test where last_name = 'Obama';
MySQL will find it from its index files, not from record 1 to last one. Of course, we should use more disk space to store the index files.
There are three ways to create index
Index for one column
alter table test add index(last_name);
Parts index for one column
alter table test add index(last_name(4));
it will save space.
Index for multiple columns
alter table test add index(last_name, first_name);
In fact, the MySQL will one execute one index for last_name or first_name. MySQL will determine which one is used dynamically. E.g. If first_name is less, first_name will be used.
invalid command
SELECT * FROM test WHERE first_name='Michael';
SELECT * FROM test WHERE last_name='Widenius' OR first_name='Michael';
valid command
SELECT * FROM test WHERE last_name='Widenius';
SELECT * FROM test WHERE last_name='Widenius' AND first_name='Michael';
SELECT * FROM test WHERE last_name='Widenius' AND (first_name='Michael' OR first_name='Monty');
SELECT * FROM test WHERE last_name='Widenius' AND first_name >='M' AND first_name < 'N';
Unique Index for one column
alter table test add unique (last_name);
It will check the unique attribute before insert a new record
Data structure
Index will use B-Tree to create a index file.
Get all index columns
show indexes from test;
When will you should use index
1. find a fixed string, e.g. where last_name='obama';
2. join two table, e.g. on table_a.last_name =b_table.last_name
3. find a range e.g. where id > 100
4. Min(), Max()
5. order by , group by
CREATE TABLE test (
id INT NOT NULL,
last_name CHAR(30) NOT NULL,
first_name CHAR(30) NOT NULL,
PRIMARY KEY (id),
);
If we execute
select * from test where last_name = 'Obama';
the MySQL will check all records in table test. It will be the O(n) questions.
If we use index for this table, it is not necessary to check each row from 0 to n. How to improve the performance to search one or more special rows in table?
For example, when we want to find a keyword in a book, we find it in the index list and then get the page no from it. We can find it quickly based on the book index. MySQL may also use this method to get it by index.
Now we use command
alter table test add index(last_name);
We have added one index on table test for last_name. if we execute
select * from test where last_name = 'Obama';
MySQL will find it from its index files, not from record 1 to last one. Of course, we should use more disk space to store the index files.
There are three ways to create index
Index for one column
alter table test add index(last_name);
Parts index for one column
alter table test add index(last_name(4));
it will save space.
Index for multiple columns
alter table test add index(last_name, first_name);
In fact, the MySQL will one execute one index for last_name or first_name. MySQL will determine which one is used dynamically. E.g. If first_name is less, first_name will be used.
invalid command
SELECT * FROM test WHERE first_name='Michael';
SELECT * FROM test WHERE last_name='Widenius' OR first_name='Michael';
valid command
SELECT * FROM test WHERE last_name='Widenius';
SELECT * FROM test WHERE last_name='Widenius' AND first_name='Michael';
SELECT * FROM test WHERE last_name='Widenius' AND (first_name='Michael' OR first_name='Monty');
SELECT * FROM test WHERE last_name='Widenius' AND first_name >='M' AND first_name < 'N';
Unique Index for one column
alter table test add unique (last_name);
It will check the unique attribute before insert a new record
Data structure
Index will use B-Tree to create a index file.
Get all index columns
show indexes from test;
When will you should use index
1. find a fixed string, e.g. where last_name='obama';
2. join two table, e.g. on table_a.last_name =b_table.last_name
3. find a range e.g. where id > 100
4. Min(), Max()
5. order by , group by
2/13/2013
use select, show databases, show tables, external sql file of mysql by shell script utility
usually we use shell to backup, update the database. Now I give one example for you to use mysql based on shell script.
Four Shell Example for:
select
show databases;
show tables;
sql.sql file
Execute one select command
> ./bash.sh host user dbnam
-------------------------bash.sh-------------------------------------
#!/bin/sh
HOST=localhost
if [ $1 ]
then
HOST=$1
fi
user=root
if [ $2 ]
then
user=$2
fi
db=dbname
if [ $3 ]
then
db=$3
fi
mysql="$(which mysql)"
read -s -p "Please input the db pass: " pass
if [ $pass ]
then
DBS="$($mysql -h$HOST -u$user -p$pass -D$db -e 'select uid from dd0a05_users limit 0, 10')"
for db in $DBS
do
echo $db
done
else
echo 'No db pass'
fi
------------------------------end bash.sh-------------------------------------------
Execute 'show databases' command
> ./bash.sh host user dbnam
-------------------------bash.sh-------------------------------------
#!/bin/sh
HOST=localhost
if [ $1 ]
then
HOST=$1
fi
user=root
if [ $2 ]
then
user=$2
fi
db=dbname
if [ $3 ]
then
db=$3
fi
mysql="$(which mysql)"
read -s -p "Please input the db pass: " pass
if [ $pass ]
then
DBS="$($mysql -h$HOST -u$user -p$pass -e 'show databases')"
for db in $DBS
do
echo $db
done
else
echo 'No db pass'
fi
------------------------------end bash.sh-------------------------------------------
Execute 'show tables' command
> ./bash.sh host user dbnam
-------------------------bash.sh-------------------------------------
#!/bin/sh
HOST=localhost
if [ $1 ]
then
HOST=$1
fi
user=root
if [ $2 ]
then
user=$2
fi
db=dbname
if [ $3 ]
then
db=$3
fi
mysql="$(which mysql)"
read -s -p "Please input the db pass: " pass
if [ $pass ]
then
DBS="$($mysql -h$HOST -u$user -p$pass -D$db -e 'show tables')"
for db in $DBS
do
echo $db
done
else
echo 'No db pass'
fi
------------------------------end bash.sh-------------------------------------------
Execute external files 'sql.sql' command
> ./bash.sh host user dbnam
-------------------------bash.sh-------------------------------------
#!/bin/sh
HOST=localhost
if [ $1 ]
then
HOST=$1
fi
user=root
if [ $2 ]
then
user=$2
fi
db=dbname
if [ $3 ]
then
db=$3
fi
mysql="$(which mysql)"
read -s -p "Please input the db pass: " pass
if [ $pass ]
then
DBS="$($mysql -h$HOST -u$user -p$pass -D$db < sql.sql)"
for db in $DBS
do
echo $db
done
else
echo 'No db pass'
fi
------------------------------end bash.sh-------------------------------------------
Four Shell Example for:
select
show databases;
show tables;
sql.sql file
Execute one select command
> ./bash.sh host user dbnam
-------------------------bash.sh-------------------------------------
#!/bin/sh
HOST=localhost
if [ $1 ]
then
HOST=$1
fi
user=root
if [ $2 ]
then
user=$2
fi
db=dbname
if [ $3 ]
then
db=$3
fi
mysql="$(which mysql)"
read -s -p "Please input the db pass: " pass
if [ $pass ]
then
DBS="$($mysql -h$HOST -u$user -p$pass -D$db -e 'select uid from dd0a05_users limit 0, 10')"
for db in $DBS
do
echo $db
done
else
echo 'No db pass'
fi
------------------------------end bash.sh-------------------------------------------
Execute 'show databases' command
> ./bash.sh host user dbnam
-------------------------bash.sh-------------------------------------
#!/bin/sh
HOST=localhost
if [ $1 ]
then
HOST=$1
fi
user=root
if [ $2 ]
then
user=$2
fi
db=dbname
if [ $3 ]
then
db=$3
fi
mysql="$(which mysql)"
read -s -p "Please input the db pass: " pass
if [ $pass ]
then
DBS="$($mysql -h$HOST -u$user -p$pass -e 'show databases')"
for db in $DBS
do
echo $db
done
else
echo 'No db pass'
fi
------------------------------end bash.sh-------------------------------------------
Execute 'show tables' command
> ./bash.sh host user dbnam
-------------------------bash.sh-------------------------------------
#!/bin/sh
HOST=localhost
if [ $1 ]
then
HOST=$1
fi
user=root
if [ $2 ]
then
user=$2
fi
db=dbname
if [ $3 ]
then
db=$3
fi
mysql="$(which mysql)"
read -s -p "Please input the db pass: " pass
if [ $pass ]
then
DBS="$($mysql -h$HOST -u$user -p$pass -D$db -e 'show tables')"
for db in $DBS
do
echo $db
done
else
echo 'No db pass'
fi
------------------------------end bash.sh-------------------------------------------
Execute external files 'sql.sql' command
> ./bash.sh host user dbnam
-------------------------bash.sh-------------------------------------
#!/bin/sh
HOST=localhost
if [ $1 ]
then
HOST=$1
fi
user=root
if [ $2 ]
then
user=$2
fi
db=dbname
if [ $3 ]
then
db=$3
fi
mysql="$(which mysql)"
read -s -p "Please input the db pass: " pass
if [ $pass ]
then
DBS="$($mysql -h$HOST -u$user -p$pass -D$db < sql.sql)"
for db in $DBS
do
echo $db
done
else
echo 'No db pass'
fi
------------------------------end bash.sh-------------------------------------------
What is difference PHP date vs gmdate
When we use date and gmdate, they may get different date based on your time zone. For example,
php code:
echo date_default_timezone_get ().PHP_EOL;
echo 'Result with date("M d Y H:i:s"):'.PHP_EOL;
echo date("M d Y H:i:s").PHP_EOL;
echo 'Result with gmdate("M d Y H:i:s"):'.PHP_EOL;
echo gmdate("M d Y H:i:s").PHP_EOL;
Result:
Asia/Tokyo
Result with date("M d Y H:i:s"):
Feb 13 2013 09:00:05
Result with gmdate("M d Y H:i:s"):
Feb 13 2013 00:00:05
Reason:
1. date is that the time returned is your time zone. Usually you may change it by using date_default_timezone_set('your timezone');
2. gmdate is that the time returned is Greenwich Mean Time (GMT).
php code:
echo date_default_timezone_get ().PHP_EOL;
echo 'Result with date("M d Y H:i:s"):'.PHP_EOL;
echo date("M d Y H:i:s").PHP_EOL;
echo 'Result with gmdate("M d Y H:i:s"):'.PHP_EOL;
echo gmdate("M d Y H:i:s").PHP_EOL;
Result:
Asia/Tokyo
Result with date("M d Y H:i:s"):
Feb 13 2013 09:00:05
Result with gmdate("M d Y H:i:s"):
Feb 13 2013 00:00:05
Reason:
1. date is that the time returned is your time zone. Usually you may change it by using date_default_timezone_set('your timezone');
2. gmdate is that the time returned is Greenwich Mean Time (GMT).
2/08/2013
Running Hadoop on Ubuntu Linux (Single-Node Cluster) for Hadoop installation
Hadoop is a framework written in Java for running applications on large clusters of commodity hardware and incorporates features similar to those of the Google File System (GFS) and of the MapReducecomputing paradigm. Hadoop’s HDFS is a highly fault-tolerant distributed file system and, like Hadoop in general, designed to be deployed on low-cost hardware. It provides high throughput access to application data and is suitable for applications that have large data sets.
The main goal of this tutorial is to get a simple Hadoop installation up and running so that you can play around with the software and learn more about it.
More please go to
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/
add user, group, show list all user or group on Centos 6
On Centos 6, you may freely create user, group. Now I give some example for these.
1. Create a group hadoop (it will be saved into /etc/group )
Usage: groupadd [options] GROUP
Options:
-f, --force exit successfully if the group already exists,
and cancel -g if the GID is already used
-g, --gid GID use GID for the new group
-h, --help display this help message and exit
-K, --key KEY=VALUE override /etc/login.defs defaults
-o, --non-unique allow to create groups with duplicate
(non-unique) GID
-p, --password PASSWORD use this encrypted password for the new group
-r, --system create a system account
e.g.
sudo groupadd hadoop
2. Create one user hduser (it will be save into /etc/passwd)
sage: useradd [options] LOGIN
Options:
-b, --base-dir BASE_DIR base directory for the home directory of the
new account
-c, --comment COMMENT GECOS field of the new account
-d, --home-dir HOME_DIR home directory of the new account
-D, --defaults print or change default useradd configuration
-e, --expiredate EXPIRE_DATE expiration date of the new account
-f, --inactive INACTIVE password inactivity period of the new account
-g, --gid GROUP name or ID of the primary group of the new
account
-G, --groups GROUPS list of supplementary groups of the new
account
-h, --help display this help message and exit
-k, --skel SKEL_DIR use this alternative skeleton directory
-K, --key KEY=VALUE override /etc/login.defs defaults
-l, --no-log-init do not add the user to the lastlog and
faillog databases
-m, --create-home create the user's home directory
-M, --no-create-home do not create the user's home directory
-N, --no-user-group do not create a group with the same name as
the user
-o, --non-unique allow to create users with duplicate
(non-unique) UID
-p, --password PASSWORD encrypted password of the new account
-r, --system create a system account
-s, --shell SHELL login shell of the new account
-u, --uid UID user ID of the new account
-U, --user-group create a group with the same name as the user
-Z, --selinux-user SEUSER use a specific SEUSER for the SELinux user mapping
e.g.
useradd -g hadoop hduser
3. show list all users
cat /etc/passwd | cut -d ":" -f 1
4: show list all groups
cat /etc/group |cut -d ":" -f 1
For cat and cut, please refer the following help
5. cat
Usage: cat [OPTION]... [FILE]...
Concatenate FILE(s), or standard input, to standard output.
-A, --show-all equivalent to -vET
-b, --number-nonblank number nonempty output lines
-e equivalent to -vE
-E, --show-ends display $ at end of each line
-n, --number number all output lines
-s, --squeeze-blank suppress repeated empty output lines
-t equivalent to -vT
-T, --show-tabs display TAB characters as ^I
-u (ignored)
-v, --show-nonprinting use ^ and M- notation, except for LFD and TAB
--help display this help and exit
--version output version information and exit
With no FILE, or when FILE is -, read standard input.
Examples:
cat f - g Output f's contents, then standard input, then g's contents.
cat Copy standard input to standard output.
1. Create a group hadoop (it will be saved into /etc/group )
Usage: groupadd [options] GROUP
Options:
-f, --force exit successfully if the group already exists,
and cancel -g if the GID is already used
-g, --gid GID use GID for the new group
-h, --help display this help message and exit
-K, --key KEY=VALUE override /etc/login.defs defaults
-o, --non-unique allow to create groups with duplicate
(non-unique) GID
-p, --password PASSWORD use this encrypted password for the new group
-r, --system create a system account
e.g.
sudo groupadd hadoop
2. Create one user hduser (it will be save into /etc/passwd)
sage: useradd [options] LOGIN
Options:
-b, --base-dir BASE_DIR base directory for the home directory of the
new account
-c, --comment COMMENT GECOS field of the new account
-d, --home-dir HOME_DIR home directory of the new account
-D, --defaults print or change default useradd configuration
-e, --expiredate EXPIRE_DATE expiration date of the new account
-f, --inactive INACTIVE password inactivity period of the new account
-g, --gid GROUP name or ID of the primary group of the new
account
-G, --groups GROUPS list of supplementary groups of the new
account
-h, --help display this help message and exit
-k, --skel SKEL_DIR use this alternative skeleton directory
-K, --key KEY=VALUE override /etc/login.defs defaults
-l, --no-log-init do not add the user to the lastlog and
faillog databases
-m, --create-home create the user's home directory
-M, --no-create-home do not create the user's home directory
-N, --no-user-group do not create a group with the same name as
the user
-o, --non-unique allow to create users with duplicate
(non-unique) UID
-p, --password PASSWORD encrypted password of the new account
-r, --system create a system account
-s, --shell SHELL login shell of the new account
-u, --uid UID user ID of the new account
-U, --user-group create a group with the same name as the user
-Z, --selinux-user SEUSER use a specific SEUSER for the SELinux user mapping
e.g.
useradd -g hadoop hduser
3. show list all users
cat /etc/passwd | cut -d ":" -f 1
4: show list all groups
cat /etc/group |cut -d ":" -f 1
For cat and cut, please refer the following help
5. cat
Usage: cat [OPTION]... [FILE]...
Concatenate FILE(s), or standard input, to standard output.
-A, --show-all equivalent to -vET
-b, --number-nonblank number nonempty output lines
-e equivalent to -vE
-E, --show-ends display $ at end of each line
-n, --number number all output lines
-s, --squeeze-blank suppress repeated empty output lines
-t equivalent to -vT
-T, --show-tabs display TAB characters as ^I
-u (ignored)
-v, --show-nonprinting use ^ and M- notation, except for LFD and TAB
--help display this help and exit
--version output version information and exit
With no FILE, or when FILE is -, read standard input.
Examples:
cat f - g Output f's contents, then standard input, then g's contents.
cat Copy standard input to standard output.
6. cut
Usage: cut OPTION... [FILE]...
Print selected parts of lines from each FILE to standard output.
Mandatory arguments to long options are mandatory for short options too.
-b, --bytes=LIST select only these bytes
-c, --characters=LIST select only these characters
-d, --delimiter=DELIM use DELIM instead of TAB for field delimiter
-f, --fields=LIST select only these fields; also print any line
that contains no delimiter character, unless
the -s option is specified
-n with -b: don't split multibyte characters
--complement complement the set of selected bytes, characters
or fields
-s, --only-delimited do not print lines not containing delimiters
--output-delimiter=STRING use STRING as the output delimiter
the default is to use the input delimiter
--help display this help and exit
--version output version information and exit
Use one, and only one of -b, -c or -f. Each LIST is made up of one
range, or many ranges separated by commas. Selected input is written
in the same order that it is read, and is written exactly once.
Each range is one of:
N N'th byte, character or field, counted from 1
N- from N'th byte, character or field, to end of line
N-M from N'th to M'th (included) byte, character or field
-M from first to M'th (included) byte, character or field
With no FILE, or when FILE is -, read standard input.
2/04/2013
Can not download csv file based on php header
It works on firefox and chrome, but IE (Normally on IE 8) fails to recognize it as a csv file (thinking it is a html fle) and when I click save I get the error, "Unable to download {name of file} from {name of site}. Unable to open this internet site. ..."
Methods:
header("Pragma: public");
header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
header("Cache-Control: public");
header("Content-Disposition: attachment; filename=\"my-data.csv\"");
header("Content-type: application/octet-stream; name=\"my-data.csv\"");
header("Content-Transfer-Encoding: binary");
?>
Methods:
header("Pragma: public");
header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
header("Cache-Control: public");
header("Content-Disposition: attachment; filename=\"my-data.csv\"");
header("Content-type: application/octet-stream; name=\"my-data.csv\"");
header("Content-Transfer-Encoding: binary");
?>
Subscribe to:
Posts (Atom)