Automatic Feature Selection in Neuroevolution - Semantic Scholar

Automatic Feature Selection in Neuroevolution Shimon Whiteson

Peter Stone

Kenneth O. Stanley

Dept. of Computer Sciences University of Texas at Austin 1 University Station, C0500 Austin, TX 78712-0233

Dept. of Computer Sciences University of Texas at Austin 1 University Station, C0500 Austin, TX 78712-0233

Dept. of Computer Sciences University of Texas at Austin 1 University Station, C0500 Austin, TX 78712-0233

[email protected]

[email protected]

[email protected]

Risto Miikkulainen

Nate Kohl

Dept. of Computer Sciences University of Texas at Austin 1 University Station, C0500 Austin, TX 78712-0233

Dept. of Computer Sciences University of Texas at Austin 1 University Station, C0500 Austin, TX 78712-0233

[email protected]

[email protected]

ABSTRACT



   
 
    

       

          
 




 
 



     
   
       

  

  
    

 
 
    



 
  
 

 


   


 


 
 

       
   




   
 
  
  !"  

 
!" 

   
      

 
 



 

      

  
 
 # 
  

 $  % 

 %  
    
%  !" 

 




   
    
   

   

  &  



 
             
  
   !"  
 

  
   
  !" &   % 

  
 
 

  
' 


   
 
%  !"$
  

   

  




    
  

   Æ Categories and Subject Descriptors &() *
     +, -
 .



  

General Terms !
 
  Keywords 

     % 
 
 % 



 

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. GECCO’05, June 25–29, 2005, Washington, DC, USA. Copyright 2005 ACM 1-59593-010-8/05/0006 .../5.00.

1. INTRODUCTION   

  
  

  
%   
 
  


 
   
   
 
% 
  
' 
 

   


    
  % 

  

  %  
   


   
   
 
 
  
  
   

    
     
 

  
     
 
 0

% 
      

  

   




  '
      
 

  
 
  
  
  

      
      


 


 

' 
  
 

  
  '
  
  
 

   

  
  
  


 1
 

       

   
    
  
 &  
   % 

  
   %  

%    

  

       ' 





   


 
  

     
  

    

  





 
   

 
  
    

    

  
 &    

 


%   
  
  

    1 

 %   
2      


   

     
 
 
   

  
%

  


 

  

 
 0 

% 
 
'

  

  



  
    
 3  


$
 
 &        
 4
    

      5% 


   

 

  
 



   
 

 
      




 
  

 
 0 

%   
   %  
 

'  
 



 


 
 %
    
  


 
  
    &    %    



   

 -
 *)+   
 



  
  '
   
 
,        
 *6% 7+  8


  


    
  

          
 


 &
% 
 
  

  
  8
  

 
  


 


  
 
   

 
 

  

% (997


 
  
       


 

  
   

   

 
      
 
 
   & 
 

 
  
 % 
  

    
% 
  
  '
 
  

 
 #  % 


 *:% ;+ 
  


  

    
 
 
       
  
  
 -


 
 
  

    

   

  
 

 
    
 0 

%  
' 
  


  
    
 


 


<
        

' 
 
 
 

         
 

 
%   

       
 




 

 !    "
  

  
 4  !"5   
 
  
                




   
  

    !" 

   

 


  

 
    
    


 


    0

%  
   

  



 
% 

  
%  

   !"  
  

 !    "
  

 
 4 !"5     *69+%  
 
   


   
    
 
  !" 
 
       
  

   
  
 = 
  

   
  *6>+
 
 

    
    
 

 % !"   
  



 


       

    

 
, !" 
  
 

   
     Æ 
 

 
   *69% 66+  !" 
 


   
 !"  
  

 $   

 ? 
  %       
 

 
 

 $  % 

 %  
  0

  *>+  
 

     
 
   


   %  
 !"% 
 
   
' 
     
  
 8
   
  
 & *6@+% 

  

    !"  


        
     
 
 
 
    
    


    
  

   
   
 

 
   
 "    

 !"     
    
       

  4 
    
  
      

  
  
 5 "
  

 

% !"     
  
        
  
       

 

 
  

 
%  
    
     
2  
      
  !" 
  
'  =
 
   

  
%  

  8  
   
 
  
  
<  3    
  
 

 
 

 

  !" 

     

 
 & 
    

   
    


 
    
 !" 


       
    4
 

        
5  
 
 
 

       !   


 

   
 !"%   
  
    
     



          
 

    
     
 

 
 &           
 
 %   






 
      !"  
 !"    
  
 

 
    ,    

        
 A"A *6(+ 



 



 
     
  
 
  
  

  

%  !"  
 

  
   
 !" &   % 


  
 
  

   !"
 
 

  
' 


   

 
    
   8
     
   (
   
!"      
    
   
  !" 
  @  
 
A"A     

 

  

 
      !"  
 !" 
  >  
 
     


  
  7  
  


 
   
  2. METHOD   
    
    
!"     &   
  !"%   
      


   !" 
  2.1 NeuroEvolution of Augmenting Topologies &     % 


  

     
     
 
     =    

   
  '
 *6>+%   
 
    
   



  

 
 
 8
 

  
% 
  
'

  
     



 
   


  
 
 

   

   
      ' 
 !    "
  

 
 4 !"5 *69+   

   
  '
  
   
' 
 
 
  


 

    
 &
%    
 
 


    
  
   
 
 !"   
 
 
   



 

  
      
     

  
   

    

 
, !" 
  
 

   
 %
  

   

   *69+ &   % 

!"      

  
  

  
 
 

 %   
      


 
   
   


 
  %  
 
       

     *66+ 




 
 
!"   

   
   
 
  &   
 % 
!" 
   
2 


<   
  

 

    
!" 
    
  *69+ 2.1.1 Genetic Encoding with Historical Markings

!   
  

' 
  2
 


 
    ! 
 
 !" 
     
 
 %
    

   
  
  

 !  
  


 
 
 
% 


 
% 

   
 
 % 

    
 
  


 

 4

 5%   
  
  %     !"    
   

     
 B   !"   
   
  
   
  
 C 
  
  
   

   
 

   
 

   
   


       


   

 
6   
 




 %     
 
  
 

  
(   




 %     
    

      

 % 
 
     8





%  
 
     

  

  
   
 
  

   Outputs Add Node Hidden Nodes

Mutation

Inputs


       


       
       
      
          Outputs Add Link Hidden Nodes

Mutation

Inputs


       
! 

      
        
"      " 

     & 
 
    
% 

  

 
   

   


      


      
% !" 

   
      

 

 =


  
 




 4     5%  
 
 
   


   
    

 
    
   


       

 

 

 =


 


 
   
%    
  
 
 

 



  % 
      

 

 

       
         
% 

  
   

        D

       


 
  % 

   

 
        


 

 
$     
 =
    
% 


    
 
   

   
 
 
   D

       

 
   
 



%   
 

' %     
    0        !" 
    
   



 


     D
 
   

   8    8
     
   
  %  
 
     

 

 
 *E+ 
 
   
 2.1.2 Speciation

&  
%   
 
  
      

  
 0 

% !" 
 
 


  %        

      


  
 
    


   
   % 

      
 

  
 


  8

 

 

 
  

 


  
  


  0      
  
  

 

  



   

 
  

       
 
Æ 


  
 
       
 
      

 

 4 5   3  4
5 

%  
  



   




   

 4 5, ½ Æ F  G ¾
G ¿  465  

Æ
 ½ % ¾ %  ¿ 3 
 
 
 

 %  
   % 

  

 




 
 
%   8
   
 
 8
 D
 
 


 
   
<   
 
$  
     
 

  


  
   Æ %         
 %   
    

 ! 
 



  
 

 

        
%     
 
   
  


 

   
     !"       *(+% 

   



   


  
  
% 

   


     
 


  2.1.3 Minimizing Dimensionality

? 

 
  
 

  

 
  
  *@% 6>+ !" 
       

     

     
  
     

  
   
 
  
 

    
 
   


  
 
  
  
  
       

 

 
  

 
%  
    
  
     
2  
      
  !" 
  
'  =
 
   

  
%  

  8  
   
 
  
  
<  3    
  
 

 
 

 

 2.2 Feature Selective NeuroEvolution of Augmenting Topologies !"$    
  
         
 

 
   
 %   
     

  
 
  % 
2
         
 
  

    
 

  

  

  &    


  
  

 


   

%        
 
 0 

%       

   
 
  
      
   

 

 &  
% 
     
  
  
 !"     
  
 

  
  
 8
 

  
  !"  

   !"   
    
   
   

  
  ,   
            "  
 !"% 
  
    

         
      
 

 
   
 0

%  !" 
  

  
  
   
   
 !"  



      ,

  
 

    
     
 

 


!    
   8



   
   

   
H 
        



  
      
  
      0 

%  
      


   1  % 
   

  



    
    

 

 
%  
 
    
  

 %  !"   

              
  
 
@   
 
   
  

 
  
 !"   !" "
 
  

   

 
%  !" 


  

 !"*69+ &   %  !"$    
     



 
  
 0 

%  
   
  
  

         
  


  


 
    

  
    
  

   
  C 


   
  
   
   
    

  



 
 % 

   
  Outputs




 
 %     

 
 


    
  


    
   
   
 

  
   



   
 0   

 
 
  
 

  
 

 

  

 
H & 


 

% 

  !"
 
     

      
 
 & 

 

% !"   
 
  
   

  

  
%      


 
  
 
     !"     
    
% 

  
  
    
  ;9 

 4

   
   
6;9 



    
5%   


  
 
  

  &   %  
 ;9 

   

%
    


   
       

*9 6+ % 

 
  
   


$ 
 
    
   %       6)(   &  !"       
  
 
  

 %    
  
 !"%     
  
 6)(

Inputs Regular NEAT

FS−NEAT


#      
   
"
 
    $!  
 
  
       %   !      
&       $!   
 %       
 &              
 !"   
 




    
  

   
%    


  



 



 
  


 &   %  
 
   
  

   

  %   
 


  
 

 
   
 

 
  
 
 
 

   3.

EXPERIMENTS 


 
 


    
 

 
  
A   " A     4A"A5 *6(+%  I 
     
    
   
    
       
    ' 

    
   

         &   % A"A  
 
 
    
 

  
 
% 

Æ
             


     


  
 
 
 
 
          



   
    
    
  
  '    
      
A"A    
  
       
$ 
 

  
         
   

 
%    
>%  

 3
    

 
 
 
 
$ 

  

 


 
  
 


' 
   
 
&   ( ($   
& ) %    * 

+    ,
              
   
 %   
%  &     

 % 

  
 

, 

   

$ 
 
 

  

 
   

$ 
 

  & 

 
%        (999  

  
 
 J %K    
7     


 

  
    
Æ

      
   

     

 4 
    
 % %    
 
5  
  

%     

  
  
  

      
 

   
  
 
 

  


  

  3 


$
   

 " 

 
     
   
   F ( % 

  
 


    

   
 
  A"A      
 




  
  

     %
 

   !"     69  < 

$ 
  


 
 
 


  

  





    
    
!" 

 
  
  



 
 ?    
 % 

 


 
     
 !"   !" 
 
 % 
 
 69 %
       (99 

  

 
 8
  
) !  
 
 

 
 
 



  

 
   
 

 % 

 
  69  
 
 
   
  
 
 
  

  

%  !"  
 

 
   


Score of Best Net Per Generation 18000 16000 14000

Scores

12000 10000 8000 6000 4000 2000


-  ./ 
     $! 


   
  
   
%     )  
    
    $!   
% * 
 
   
 !" &       





 
 %  
$ 
 
 
%   E7L   

% 
     
 
 





  !"  
 !" 
:  %   



 
%   

 
 
 
 
 

    
  

 
  
 

  A
 !"  
  6)(     !"  

 
    
      
 & % 
  !"$


 

   
   

  )7% 
  
 
 6:7L 

   
 !"

 

%      
 
    

 69L    

    !"$
 
  
  

   
 

  )7%      
 >)L  

  (99%      
 
    

((EL      
 !" 
;  %   



 
% 
 8
 

 
   
 

 % 

 8
    
  
   
 4   

   
 
5

 
  
       
 
    !"
 
   
 
    
 !" 
 =
  !"$
 

   
   

  )7%  
 
    

 E:L  
 
 !"$ =
 
   

 

  (99%  !"$ 
 
    

 6;7L  
 
 !"$ & 



 
%  !"   
   
    
 6)  %    
   



 



  


%   ;9 

%   
   


 0    

 


  !" 
 
 
 !"  
 
   


 

 
  
H 0   
   

 

  


 
 
 !"   
     
H 
  !"$
 


 

   
 

 



 
 
H  
 

'
 % 
 
 

   

 
   


   

  8
 





 
 


 
 
  
% 
  ;9 

% 
 
7% (9% >9%  6)9 

 &
 
% 


 

   
'

FS-NEAT Regular NEAT 0 0

50

100 Generations

150


0 
   

  
 ! 
    $!    ( ($   0 %"    12   


 %   

     


   

!  %  "&  "  
 
  
  %
  %
 2
 3&  
  

!   
  $!   
   & " ! 

    
   

 
  
  

   "  

        
    


   8
6(% >(% ;(%  @((


  8
  
 
 % 
 
 69 %
       (99 

  
E  8
 

  



 
    %  
 
   


  8
% 

  
 

 
   

 
% 

 
  
  !
 
 
   


     6(  %  !"  
  

 " 
 8
 



  % 

 
 
 !" 

 


 #  % 

 
  !" 
  
  

  




    
 
 


  
 Æ 
69   
 

   

    

 
   

 
% 

 
  
  A
 !"  
   
   
 !" 
  
 
 
 !
  
 8
 



  % 

    
   !"$ 
 
   
     % 
66   
 
 8
  



  
 8
 
 !"$ 
 
  

  
   

  

   


% 

  !"$ 
 
   
     8
 

  
%  !"   
 
 

 
    



     
 !" %     %  

    
 4. DISCUSSION 

   
 


    
 
 
  
  
 
 
  

  

%  !"  
 

 
   
 
 
   
 !" &   % 

   
 

  


   


 


200

Connected Inputs of Best Net Per Generation

Size of Best Net Per Generation

180

800

160

700

140

600

500 100

Sizes

Connected Inputs

120

400

80 300 60 200

40

100

20 FS-NEAT Regular NEAT

FS-NEAT Regular NEAT

0

0 0

50

100

150

Generations


4 
   "
   

  "&
 
    $!    ( ($   0 %"       




   "
           
  
     "  ! 
   
  (  
  &    0    "  $!  %%  " 

      &  
     
   


   

  8
 1


  '
   
  


    
 8
 
    
   !"   


 
 
% 
)    
 


 

   

   

  )7 0 

%  
 :  ;     
 8
 
    
   !"$ 
 
    
    
    

  (99  $ 


 
  
 



J  K  8
 

  H C 
   
% 

    
     !"   !"   

 

     
    
  
  0

% 

 
  
    
  
% 

     

 
  
 
   
 
 & 



 
%  
     
   
 
 

  )7




%

  
 
   % 
   

  

 8

  



   

 




  
 
   
 
 


      
 
 


     
   
   
    !"      

   
  
  

     
  
 







       
 %  
 
% !"  
  
 

 

    
  
 8
  &   

%  !" 


  
    ,  

   

   
 
  8


       
 


 &
   
 %        

  
  

  
 
 
, 
  
    
 
      
%  
   
   
     
   !"  
   

      




    
    

   

 = 



 
 


    
 
     !"   
    

200

0

50

100 Generations

150


1 
   5   


%%  "&
 
    $!   

( ($   0 %"     

  


   "
   ) &       
  *    "
     "  
   
  $!   %%    & 

 
 
      
   
 %  


 
 

  


   8
, 699 =
 

  

 

  
  !"   

 
  


    

               
     


 0 

%    
 
     
A"A    & % 
  

 
    



   8
  
    
 !"   !"
     

     (7     !" 
     



 
 !" 0

%  
  A"A%  !"$ 
    
  

 
  
  
    
    
  
  

  
   !"$  
   
 
 
 
 
 8
 
   


 

 "   
 
 
% 



  
  
  
 %   
  

 
 
  !"$
 
 
E  !"$
  
%   % 
   
 
  B    % 
 8
 
    
   !"$ 
 
   
  

  
      

 
 

  8
% 

 
 !"$ 
   

 
  

% 


 
   

Æ   !" 
 
  




   
 5. FUTURE WORK "   '
      
 
   
   


  

       
    
 
   


 
 


    
  

 
     

 

   !" 
 
 

     0 

%   
 

  
 

  
  


 
  

 
 
  %  
A"A   % 

 

 
 
    
 
 &   %   
$

 
    


200

Score of Best Net Per Run

Connected Inputs of Best Net Per Run 350

21000

FS-NEAT Regular NEAT 300

19000

250 Connected Inputs

20000

Scores

18000

17000

16000

200

150

100

15000

50 FS-NEAT Regular NEAT

14000

0 0

50

100

150

200

250

300

350

0

50

Number of Available Inputs

100

150 200 Number of Available Inputs

250

300


6 
   

  
!


    $!  
  
   ! 
 5      


  



 %  "&  "  
   

 %
  %
 2
  

  
 ! 
      & 
    
    


  

   $!  & 
&   


2 
   "
       
 
    $!  
  ! 
   ,
 5      


   "
        

    "  
   

 %
  %
 2
 (  
  &

   %"     $!   
 

   & 
 " 

    

    
  
       
  0 

%     
%   
       

  &  
%    
 
 
  

 
 
  % 
   
   


      
  

   

 -
  

 % 
 

  
 

 

    

  
 %    

  
    

 
 0

%  !"   
  
   
    

  
   
  

   
 
 
  
 1  
% 
 !"   



         0 

%  
    
  
 
 
 
 % 
 


     
 
  
   
   
 
 
% 
 


     
  

  
         
 
 
 
!"$       
 %    
   

     
 

   

 


  

        
  


    
  


    
 
 
      

  
% 
    

  8
 
 

    

 


   
 
      



 
     

   8  
    8
  

 
  
     

  
 
 
%  

  
 
 


  
%    8   8
 
      &   %  


    
   
   %   


  
   
 
    
  & 
 % 
   

  
 
     
 
  
 =
 
  

  !"      
  
 

 
   
 



  



 
Æ

6. CONCLUSION 
         
   !"%   


   !"  

  
      



  ? 

 



  
 

%  !" 
   
 

 


   

  !   
    
            
 
  
  
 
  
  

  

%  !"  
 

 
   
 
 
   
 !" &   % 

     
 

  


   !
 
     


   

  8
      !"$
 
  

  




   
  
   
   Æ # 
  


   

    

 



 


% 



   !"   



      
!" 
   


  


 Acknowledgments   

  


           ?"%  C"A!!A  &&9(@:)EE%   &#B    APPENDIX A.

NEAT AND FS-NEAT SYSTEM PARAMETERS 


   8
 699 ! 

  
 ' 
 69  % 
  6999  
 

  

Æ
   
        

½ F 69% ¾ F 69%  ¿ F @9 
       
 Æ F @9 
   
 

    
  

   


  

 

   
 



350

*@+  D%  = 
%  - 
 "      


 

     

     

  
 
  & I A O 8%  ! D 
%  # 
%  A - A  %
  %

Size of Best Net Per Run 1400 FS-NEAT Regular NEAT 1200

 
  !!)* 
 
  +  %
 % 
 ;6N;E B& 
% 6EE)

1000

Size

800

600

400

200

0 0

50

100

150

200

250

300

Number of Available Inputs


 
  
 5 
 
    $!  
  
   ,
 5      


   5   "  
   

 %
  %
 2
 (  
  "  
  


 5   &   "
 %"  ! 
 
 
 $!  "  
 & 
&     5  

   
 997 
        
  
 99(  
      
       96 B. REFERENCES *6+ # M # 
  "  =

 

     
          
  
 
   & 
 
  !!"

#  
 $

 %  & &
' #$%&&(!"% 
 >(N79%  %  %

6EE> *(+  ! D 
  I A    D

                 

  8  &


 
  $
 #  

 
  % 
% 
 6>;N67>% 6E;:

350

*>+  A 0
%  B #

 %  I  # 
 !   


 


   
     

 & 
 
  ,--,
 
 

 
 
% 
 6;:;N6;;@% (99( *7+ O O   - A

 "   

  





  & 
 
  . #  

 
 /  0 % "
% B 
% 6EE( B  O *)+  -
 

   

 

   

  & 
 
 %%%# +

$

 1  % 6EE> *:+  B 
  O  "           


 

  # .  


 % (),E6:NE((% 6E:: *;+ I    %   %  I O 
    
 
   



     1

 0 % 67,666EN66(7% 6EE> *E+  I A 
 D

  
 
      

    
 
  

 

  8  & 
    
% 6465,):NE9% 6EE@ *69+ O 1 
  A B  
 !   
 
     
  

 
 

 
 
% 694(5,EEN6(:% (99( *66+ O 1 
  A B  
 C 
  

     
     
   2
 
 %  #

 1 % (6% (99> &


 *6(+ B !    
        % 6EE7

   
 *6@+  = 
 % O 1 
%  A B  
 "   



   

   &

  ,--"* 
 
   



 
 

  3
'

 $   4 
% I (99>

*6>+ P Q  !      
 
  
 
  #% ;:4E5,6>(@N6>>:% 6EEE