The paper investigates by numerical modeling the effects of crystallographic texture and grain shape on the shape of the yield surface of aluminum sheet material at small strains. Different representative volume elements (RVEs) of the material are considered. Plane stress state is assumed in the sheet. A rate-dependent model of crystal plasticity (CP) is used in combination with either the full-constraint (FC) Taylor model or the finite element method (FEM) to compute the volume averaged stress of the material. The effect of different crystallographic textures observed in aluminum alloys on the shape of the yield surface is firstly investigated. An analytical yield function is used to generate yield surfaces for the different crystallographic textures. The deviation between the stress states at yielding computed by FC-Taylor model and the analytical yield surface is used to evaluate the capability of the yield function to fit the anisotropic yield surfaces representing different strong crystallographic textures. Two different shapes of the grains are introduced in the RVEs of CP-FEM in order to study the effect of the grain morphology. Small effects of grain shape are found at small strain compared with the marked influence of crystallographic texture.